Cache Subsystem

Motivation

Accessing the DDR2 SDRAM incurs a significant latency, even with the pipelined MIG interface, a single 128-bit read takes on the order of 20 clock cycles. Since the CPU issues a memory access nearly every cycle (for instruction fetches and data loads/stores), directly accessing DDR2 would stall the pipeline almost constantly, reducing performance to a small fraction of the 75 MHz clock’s potential.

The cache subsystem bridges this gap by keeping frequently accessed data in fast on-chip BRAM. With an 8 KB (later increased to 32K for optimization), 2-way set-associative cache, the hit rate for instruction fetches is extremely high (code is dense and exhibits excellent spatial/temporal locality), and the data cache captures frequently accessed variables and rendering data.

Shared Memory Arrays (cache_memory_subsystem.sv)

Both the Instruction Cache and the Data Cache use the same parameterized memory array module. This module implements the tag, data, and metadata storage for a 2-way set-associative cache.

Organization

flowchart TD
    ADDR["CPU Address<br/>[31:0]"] --> SLICE{"Address Slice"}
    SLICE --> TAG["TAG Bits [31:index+offset]"]
    SLICE --> IDX["INDEX Bits"]
    SLICE --> OFF["OFFSET Bits [1:0]"]

    IDX --> WAY0["Way 0"]
    IDX --> WAY1["Way 1"]

    subgraph WAY0["Way 0"]
        TA0["Tag Array"] --> TC0["Tag Compare"]
        DA0["Data Array<br/>128-bit wide BRAM"]
    end

    subgraph WAY1["Way 1"]
        TA1["Tag Array"] --> TC1["Tag Compare"]
        DA1["Data Array<br/>128-bit wide BRAM"]
    end

    TC0 --> HIT0{"Hit?"}
    TC1 --> HIT1{"Hit?"}
    HIT0 --> MUX["Output MUX"]
    HIT1 --> MUX
    MUX --> CPU["CPU"]

Geometry

Address Slicing

localparam int OFFSET_BITS = $clog2(LINE_SIZE_BYTES / 4); // 2
localparam int INDEX_BITS  = $clog2(SETS);                 // 8
localparam int TAG_BITS    = 32 - OFFSET_BITS - INDEX_BITS; // 22

assign index_in = addr_in[OFFSET_BITS + INDEX_BITS - 1 : OFFSET_BITS];
assign tag_in   = addr_in[31 : 32 - TAG_BITS];

Memory Arrays

Each way contains three arrays indexed by the set index:

• Data array: logic [127:0] data_array_way0 [SETS], inferred as BRAM, 128 bits per line
• Tag array: logic [TAG_BITS-1:0] tag_array_way0 [SETS], stores the upper address bits
• Valid array: logic valid_array_way0 [SETS], single bit per line, reset to 0 on startup
• LRU array: logic lru_array [SETS], 0 = way 0 was least recently used, 1 = way 1 was LRU

Read Operation

On each cycle, both ways’ data and tag arrays are read speculatively using the current index. The tags are compared against the incoming address tag in parallel:

assign hit_way0  = valid_out_way0 && (tag_out_way0 == current_tag);
assign cache_hit = hit_way0 | hit_way1;

Both ways’ data are available, and the outer cache controller selects the correct one based on the hit result and the word offset (adr[1:0]).

Write Operation

When a cache line fill occurs (on a read miss), the controller asserts we_way0 or we_way1 to write the incoming 128-bit line into the appropriate way. The data array is written with byte-level masking via write_mask:

for (int i = 0; i < LINE_SIZE_BYTES; i++) begin
    if (write_mask[i]) data_array_way0[index_in][i*8 +: 8] <= write_data[i*8 +: 8];
end

LRU Replacement

On a cache hit, the LRU bit for the current set is updated to mark the other way as LRU:

if (update_lru) begin
    lru_array[index_in] <= (accessed_way == 1'b0) ? 1'b1 : 1'b0;
end

On a miss, the eviction candidate is the way indicated by lru_array[index_in] for that set.

Performance Impact

The caches dramatically reduce the average memory access latency:

For DOOM’s code, the instruction cache hit rate is very high; the game’s inner render loop fits almost entirely within the instruction cache. The data cache hit rate is lower but still provides a significant improvement for frequently accessed globals and rendering intermediate values.

The Instruction Cache and Data Cache controllers are each documented in their own subchapters.