4.6. Cache

Although retrieving data from memory is much faster than retrieving data from hard disk drives and tapes, it still cannot keep the processor pipeline full. Cache is implemented to help keep the pipeline full and the processors processing data efficiently.

Like memory, cache stores data in grids. Instead of using capacitors, however, the grids in cache are made up of latches. Because the latches do not have to be refreshed, cache can read and write data much faster than memory.

Cache works off the principle of data locality. When a processor requests data, there is a high probability that the next request will be for data stored in close proximity to the initial data that was requested. When a processor requests data from memory, memory sends the requested data plus the nearby data. All data is then stored in cache. Therefore, subsequent data requests can usually be filled from the faster cache rather than from the slower memory.

4.6.1 Cache Levels and Placement

Cache was originally located on the system board between the processor and memory. It was connected to the processor through the system bus, as shown in Figure 4-16. Eventually, cache was incorporated into the processor. Current processors usually contain more than one cache. The first-level cache is called L1 (Level 1), the second-level cache is called L2, and so on.

Figure 4-16. Cache levels and placement.

L1 cache is the fastest of all the caches, but it holds the least amount of data. Because it is connected directly to the processor, it operates at the same speed as the processor. It can deliver data in as little as 10ns.

L2 cache was also originally located on the system board and connected to the processor through the system bus. That meant it was slower than L1 cache, but still faster than memory. Now all processors used in ProLiant servers include L2 cache as part of the processor. The L2 cache is connected to the processor through a backside bus. The backside bus speed depends on the processor, but normally runs at the speed of the processor.

When L2 cache moved to the processor package, cache on the system board became known as L3 cache. L3 cache runs at the speed of the system bus, which is still faster than the memory bus. The Itanium processor now has L3 cache located on the processor. This opens the possibility for an L4 cache on the system board.

Each level of cache is bigger than the last. L2 holds a copy of everything in L1, and has room for additional data. L3 holds a copy of everything in L2, plus additional data. L2 and L3 can deliver data in about 20 to 30ns. In contrast, main memory delivers data in about 60ns because it runs at the speed of the memory bus. In comparison, hard drives can take as long as 12ms to return data to the processor.

4.6.2 Cache Lines and Tag RAM

Although DRAM is generally designed with only a few rows to make refreshing easier, SRAM does not have that limitation. As a result, cache is usually designed with many rows and only a few columns.

Example

A 512KB cache has 16,384 rows (or cache lines) and 32 columns.

(16,000 x 32 = 512,000)

Each row in the cache is known as a cache line. In newer processors, when the processor requests data, main memory returns the single byte of data requested and it also prefetches the next 31 bytes. The 32-byte chunk is stored in a cache line. This is known as a cache line fill. If the processor requests data from the additional 31 bytes (which it often does), it can retrieve the data quickly if it is already in a cache line.

A smaller SRAM chip, called the tag RAM, stores the memory address for the data in each cache line. When the processor requests a piece of data, the cache controller compares the address in the request with the addresses in the tag RAM. If the cache controller finds the address, it returns the associated data to the processor. This is known as a cache hit. If the controller does not find the address, it is known as a cache miss.