Data Cache

Controller Architecture

The data cache controller (data_cache_controller.sv) handles both load and store operations between the CPU and DDR2. Like the instruction cache, it wraps the shared cache_memory_subsystem module. However, the data cache has additional complexity because it must handle writes, byte-level write masks, and a write-through policy.

flowchart LR
    subgraph CPU
        LSU["Load/Store Unit<br/>(Memory Stage)"]
    end
    subgraph DCACHE["Data Cache Controller"]
        EVAL["Hit/Miss Eval"]
        MEM["Cache Memory Arrays"]
        FSM["FSM<br/>Read/Write Handling"]
    end
    subgraph DDR2_PATH["Memory Subsystem"]
        ARB["128-bit Arbiter"]
        BRIDGE["MIG Bridge"]
        DDR2["DDR2 SDRAM"]
    end

    LSU --> EVAL
    EVAL --> MEM
    MEM --> LSU
    EVAL --> FSM
    FSM --> ARB
    ARB --> BRIDGE
    BRIDGE --> DDR2
    DDR2 --> BRIDGE
    BRIDGE --> ARB
    ARB --> FSM

Write-Through Policy

The data cache uses a write-through policy: every store is immediately forwarded to the DDR2 memory. The cache is updated in parallel only if the address is already present (a write hit). On a write miss, only the DDR2 is written, no write-allocate is performed.

This design was chosen for simplicity: it avoids the need for a dirty-bit tracking mechanism and simplifies the coherence model (DDR2 always has the latest data). The latency cost is acceptable because store operations do not stall the pipeline in the HaDes-V architecture (the store buffer in the memory stage can absorb the latency as long as it is not immediately followed by a load to the same address).

State Machine

The controller uses a 6-state FSM:

stateDiagram-v2
    IDLE --> READ_EVAL: cpu_bus.cyc & cpu_bus.stb & !cpu_bus.we
    IDLE --> WRITE_EVAL: cpu_bus.cyc & cpu_bus.stb & cpu_bus.we
    READ_EVAL --> IDLE: !cpu_bus.cyc (abort)
    READ_EVAL --> IDLE: cache_hit (read, update LRU)
    READ_EVAL --> READ_DDR: cache_miss
    READ_DDR --> READ_WAIT_BRAM: mem_bus.ack (line fetched)
    READ_DDR --> IDLE: mem_bus.err
    READ_WAIT_BRAM --> READ_EVAL: fill written to BRAM
    WRITE_EVAL --> WRITE_DDR: always (write-through)
    WRITE_DDR --> IDLE: mem_bus.ack / mem_bus.err

Read Path

READ_EVAL: Evaluates the cache hit/miss, identically to the instruction cache. On hit, the data is extracted from the correct 128-bit lane based on cpu_bus.adr[1:0] and returned with ack. On miss, the controller issues a 128-bit read to DDR2 and transitions to READ_DDR.

READ_DDR: Waits for the DDR2 access to complete. On ack, the fetched line is written into the evicted cache way, and the controller transitions to READ_WAIT_BRAM. On err, the error is returned.

READ_WAIT_BRAM: A single-cycle wait state to ensure the BRAM write from the fill has completed before the next read attempt. The controller then returns to READ_EVAL, where the data is now present in the cache.

Write Path

WRITE_EVAL: Prepares the write data by aligning cpu_bus.dat_mosi to the correct 128-bit lane based on cpu_bus.adr[1:0]:

case (cpu_bus.adr[1:0])
    2'b00: begin lane_write_data[31:0]   = cpu_bus.dat_mosi; lane_write_mask[3:0]   = cpu_bus.sel; end
    2'b01: begin lane_write_data[63:32]  = cpu_bus.dat_mosi; lane_write_mask[7:4]   = cpu_bus.sel; end
    2'b10: begin lane_write_data[95:64]  = cpu_bus.dat_mosi; lane_write_mask[11:8]  = cpu_bus.sel; end
    2'b11: begin lane_write_data[127:96] = cpu_bus.dat_mosi; lane_write_mask[15:12] = cpu_bus.sel; end
endcase

If the address is a cache hit, the cache line is also updated (write-through + cache update):

if (cache_hit) begin
    bram_write_data <= lane_write_data;
    bram_write_mask <= lane_write_mask;
    if (hit_way0) we_way0 <= 1'b1;
    if (hit_way1) we_way1 <= 1'b1;
    update_lru   <= 1'b1;
    accessed_way <= hit_way1;
end

The DDR2 write is also initiated in this state. The controller transitions to WRITE_DDR.

WRITE_DDR: Waits for the DDR2 write acknowledgment. On ack, the store is complete, Ack is returned to the CPU, and the state returns to IDLE.

Interrupt Handling

On a read access, if the CPU de-asserts cpu_bus.cyc while in READ_EVAL, the controller aborts and returns to IDLE. During a miss fill (READ_DDR), the cycle continues regardless; the fill completes and the line is written to the cache. This ensures cache consistency even if the original requestor has moved on.

Key Differences from the Instruction Cache

Hit Latency

Same as the instruction cache: a read hit completes in a single cycle. A write hit takes two cycles (one to issue the DDR2 write, one to wait for acknowledgment), during which the cache is updated on the first cycle.