DDR2 Memory Subsystem

Motivation

DOOM requires far more memory than the HaDes-V’s on-chip BRAM can provide. The WAD file alone is 4 MB, and the game requires additional space for the code, runtime data structures (zone allocator), and two full-resolution framebuffers. The Nexys 4 DDR’s 128 MB DDR2 SDRAM provides ample capacity, but integrating it requires bridging a significant interface complexity gap between the CPU’s 32-bit Wishbone bus and the DDR2’s high-speed, pipelined native interface.

Memory Interface Generator (MIG)

The DDR2 subsystem is built around Xilinx’s MIG (Memory Interface Generator) 7-Series IP, configured for the Nexys 4 DDR’s specific DDR2 SDRAM:

The MIG abstracts the complex DDR2 physical-layer protocol, including DLL calibration, DQS strobe alignment, write leveling, and periodic refresh, behind a simpler application interface. The 200 MHz reference clock is generated from the system MMCM, and the MIG outputs a synchronous 75 MHz ui_clk that serves as the main system clock for the CPU and caches.

No-CDC Architecture

The CPU, caches, and all peripherals run directly on ui_clk, the MIG’s user interface clock. This eliminates any clock domain crossing between the memory controller and the CPU core:

// No CDC: The CPU runs on the clock generated by the memory controller
assign sys_clk = ui_clk;
assign sys_rst = ui_clk_sync_rst;

This design choice simplifies the bridge logic substantially: the Wishbone protocol operates in a single clock domain with the MIG’s native interface, removing the need for asynchronous FIFOs or handshake synchronizers on the data path.

MIG Native Interface

The MIG exposes a synchronous application port with the following key signals:

• Command: app_cmd (3-bit), app_addr (27-bit), app_en (strobe)
• Write Data: app_wdf_data (128-bit), app_wdf_mask (16-bit byte enables), app_wdf_wren, app_wdf_end
• Read Data: app_rd_data (128-bit), app_rd_data_valid
• Flow Control: app_rdy (command ready), app_wdf_rdy (write data ready)

The interface is pipelined: commands and write data can be issued on different cycles as long as the respective ready signals are asserted. Reads complete multiple cycles later with app_rd_data_valid.

128-bit Wishbone-to-MIG Bridge (wishbone_128_ddr2_bridge.sv)

The bridge translates between the 128-bit Wishbone master (from the cache subsystem) and the MIG’s native application interface:

flowchart LR
    subgraph FPGA
        direction TB
        WB["128-bit Wishbone<br/>Master (Cache)"] --> BR["Bridge"]
        BR --> MIG["MIG 7-Series"]
        MIG --> DDR2["DDR2 SDRAM<br/>128 MB"]
        BR --> WB
    end

Address Translation

The conversion from 32-bit word addresses to the MIG’s byte addresses follows this path:

logic [31:0] relative_word_addr = port.adr - ADDRESS;
logic [31:0] relative_byte_addr = relative_word_addr << 2;
assign app_addr = {relative_byte_addr[26:4], 4'b0000};

The 128-bit data path means each MIG access covers 16 bytes (4 * 32-bit words). The lower 4 bits of the byte address are cleared and the 27-bit app_addr selects 16-byte aligned blocks within the 128 MB space.

Bridge State Machine

The bridge operates in three states:

stateDiagram-v2
    IDLE --> WAIT_WRITE: Write & (!app_rdy or !app_wdf_rdy)
    IDLE --> WAIT_READ: Read & app_rdy
    IDLE --> IDLE: Valid access, both ready
    WAIT_WRITE --> IDLE: Both cmd & data accepted
    WAIT_READ --> IDLE: app_rd_data_valid

Write path: The bridge issues the command (app_cmd = 3'b000) with write data on the same cycle. If the MIG is not ready for either, it transitions to WAIT_WRITE and retries until both cmd_accepted and data_accepted are true.

Read path: The bridge issues the command and waits for app_rd_data_valid in the WAIT_READ state, then captures the data, asserts Wishbone ack, and returns to IDLE.

Byte masking: The wishbone_sel signal is inverted to produce app_wdf_mask (MIG masks are active-low), ensuring only the targeted bytes are written.

128-bit Wishbone Arbiter (wishbone_128_arbiter.sv)

The arbiter manages access to the DDR2 from two 128-bit masters: the Instruction Cache and the Data Cache.

// Priority: Data Bus > Instruction Bus
if (m1.cyc) begin
    grant_m1 <= 1; // Data bus has priority
    grant_m0 <= 0;
end else if (m0.cyc) begin
    grant_m0 <= 1;
    grant_m1 <= 0;
end

Data bus accesses are prioritized because a stalled data access (e.g., during a load or store) freezes the pipeline immediately. Prioritizing data accesses minimizes the time the CPU spends waiting on data hazards. Instruction fetches can generally tolerate longer latencies due to the ICache’s prefetch behavior.

Memory Map

The DDR2 occupies the upper portion of the 32-bit address space:

The linker script and bootloader cooperate to ensure that code and data are placed in the appropriate memory regions. The WAD is stored in DDR2, framebuffers are allocated from DDR2, and the DOOM zone allocator manages DDR2-backed heap memory. It is to be noted, that further improvements could have been possible by placing certain pieces of code or data in BRAM for faster access, but evaluating this would have been challenging due to the limited BRAM size and the complexity of the memory access patterns in DOOM.