Caches Are Faster Than Main Memory: Essentials
You probably know by now that cache memory is faster than main memory. But have you ever wondered why? The speed difference between cache and main memory is influenced by several factors. These include the proximity of the memory to the processor core, the bus width, the complexity of the data transfer protocol, and most importantly, the technology behind each type of memory.
Static Random Access Memory (SRAM)
The Core Technological Difference
While proximity to the processor core, bus width, and data transfer protocol all play a role, the technology has the biggest impact on access latency, so let's focus on that. Caches use static random access memory, also known as SRAM, while main memory uses dynamic random access memory, or DRAM. Each SRAM cell stores one bit using six transistors.
Structure of an SRAM Cell
Four of these transistors form a pair of cross-coupled inverters, essentially two NOT gates, where the output of each gate feeds into the input of the other. These inverters reinforce each other, which keeps the binary state stable as long as power is supplied. The remaining two transistors are used to read and write the stored value.
Speed vs. Size Trade-Off in SRAM
This relatively large structure takes up a lot of space on the memory chip, which limits the number of cells that can be included. As a result, SRAM is more expensive to manufacture per bit than DRAM, which is why caches are much smaller in size compared to main memory. However, the relatively simple design of the SRAM cell allows it to be accessed directly and much more quickly than DRAM.
Dynamic Random Access Memory (DRAM)
How DRAM Stores Data
DRAM, on the other hand, uses one transistor and one capacitor to store and access a single bit. The bit value is stored as a charge in the capacitor. For example, a charged capacitor means that the value of the bit is 1, and a discharged capacitor means that the stored value is 0. The transistor is used for accessing the value, either for reading or writing.
Cost and Density Advantages
Because DRAM uses only one transistor per bit of storage, it is much denser and cheaper to manufacture. But while cheaper and denser memory is great, it comes with a few major downsides. Capacitors naturally lose charge over time and must be refreshed.
Why DRAM Needs Refreshing
To prevent the stored value from being lost, each DRAM cell must be read and rewritten every few milliseconds. This is why it's called dynamic RAM in contrast to the static nature of SRAM. As you might expect, the refresh process can slow down memory access because the stored value cannot be accessed while it is being refreshed.
Sense Amplifiers and Read Complexity
Even though DRAM can refresh multiple cells at once, the refresh process makes access times a bit slower compared to SRAM. Reading a value from a DRAM cell also poses a problem. The capacitor's charge is read by a circuit called a sense amplifier.
To allow the sense amplifier to detect the charge, the electrons from the capacitor must flow into the sense amplifier, emptying the capacitor and destroying the stored value. To restore the original value, the sense amplifier drives the bitline to the detected voltage, effectively writing the data back into the cell. This entire process further increases DRAM access time.
Two-Level Decoding in DRAM Architecture
But if that wasn't enough, the actual process of finding a single bit in memory is more complex than in SRAM. DRAM cells are organized in a large matrix. To locate a single bit in the matrix, the memory uses a two-level decoder, one for row access and another for column access.
Row and Column Decoding Explained
In other words, accessing a cell is a two-step process. The address provided to the row decoder is used to select an entire row and store the data in an array of one-bit latches. The address sent to the column decoder is then used to select the required bit from the latch array.
Address Line Sharing and Timing Signals
To reduce memory chip costs, the row decoder and column decoder share the same address lines. Two additional signals called the row access strobe and column access strobe are used to indicate to the DRAM which address is currently being transferred on the lines. This complex circuitry ultimately results in the long access times associated with main memory.
Final Thoughts
So next time someone asks you why caches are faster than main memory, you'll know what to say. The fundamental difference lies in the memory technology itself. SRAM, used in caches, offers faster access at a higher cost and larger size per bit. DRAM, used in main memory, is denser and more affordable but comes with slower access due to its refresh requirements and complex data retrieval process.