The assembly line in silicon

Five stops for every instruction

Here is the loop this whole lesson runs on. It is written the way the exam papers write assembly: capital mnemonics, F registers for floating-point values, R registers for integers, and the memory address as an offset in front of a base register in brackets. The papers of 2024 and 2025 wrote the registers as $F2 and $R1; the 2026 paper dropped the dollar sign, and so does this notebook.

LOOP:  LD    F2, 0(R1)
       FADD  F4, F2, F0
       SD    F4, 0(R1)
       ADDUI R1, R1, 8
       BNE   R1, R2, LOOP

Read it once in words: load the number at address R1 into F2; add F0 to it and put the sum in F4; store F4 back to the same address; move the pointer R1 on by 8 bytes; go back to LOOP unless R1 has reached R2. The instructions are numbered I0 to I4 as the papers number them. The machine of this lesson is the one of the notes: five stages, one clock cycle each, and every instruction — the FADD included — spends exactly one cycle in the execute stage. Real floating-point adders take several cycles; that pipeline arrives in lesson 7.

The RISC-V subset of the notes, in three spellings

The notes (pp. 8–9) introduce the instruction set in three families, with the effect of each instruction written as a register transfer: ALU instructions (add rd, rs1, rs2 does rd ← rs1 + rs2; addi adds a constant), load and store instructions (ld rd, offset(rs1) does rd ← M[rs1 + offset]; sd rs2, offset(rs1) does the reverse) and branches and jumps (beq, bne, j, jr). These are the only kinds of instruction the pipeline ever sees. The same instructions appear in three spellings during the course, and you must read all three fluently, because the exam is written in the third.

Two small things to know about the first spelling. The notes use ld and sd, the 64-bit load and store of RV64, and on p. 10 call the registers 32-bit wide; in RV64 they are 64 bits (the width is called XLEN). And the notes' j and jr are pseudo-instructions. None of this matters for the exam; the mnemonics of the papers do.

What happens at each stop

Every instruction visits the same five stages in the same order. Walk LD F2, 0(R1) through them — it is the notes' Example 1 (p. 12), ld x10, 40(x1), in the papers' spelling:

1. IF, instruction fetch. The program counter (PC) addresses the instruction memory; the instruction word comes out, and a separate adder computes PC + 4, the address of the next instruction. (Separate, so that IF never needs the ALU while EX is using it for another instruction.)
2. ID, instruction decode. The opcode is decoded, the register file is read — here R1 — and the immediate 0 is extracted and sign-extended.
3. EX, execute. The ALU adds R1 + 0: the memory address.
4. MEM, memory access. The data memory is read at that address.
5. WB, write-back. The word that came out of memory is written into F2.

Now look at the hardware those five steps use, and you will see why there are exactly five of them and not four or six. Each step uses a different resource, and uses it once: the instruction memory, the two read ports of the register file, the ALU, the data memory, the one write port of the register file. The notes list these components on pp. 10–11; the register file has two read ports and one write port because every RISC-V instruction reads at most two registers and writes at most one. That one-resource-per-stop property is the precondition for everything that follows: five instructions can occupy the five stops at the same time without fighting over hardware.

Five stages because five resources: instruction memory, register read ports, ALU, data memory, register write port — each used once per instruction.

Overlap, not speed: the assembly line

In a car factory nobody waits for one car to be finished before the next body enters the line. Each station does its one job and passes the car on; a new car rolls off the end at the pace of the slowest station, not at the pace of a whole car. The notes (p. 13) use the same picture, and it is exact: the five stops of Fig. 1 are the five stations, and the instructions are the cars.

Take the notes' numbers: 2 ns per stage, so 10 ns per instruction. Without overlap, five instructions take 50 ns — each one must leave the last stop before the next enters the first. With overlap, the second instruction enters IF while the first moves to ID, and so on. The first instruction still needs the full 10 ns; after that one instruction completes every 2 ns.

Latency and throughput

Pipelining changes nothing about one instruction. Latency, the time from the fetch of an instruction to its write-back, is still five stages, 10 ns (p. 14: "does not change"). What changes is the throughput: the number of instructions completed per unit of time. One per 10 ns becomes one per 2 ns once the line is full. The notes call pipelining a technique (p. 8), and that is the right word: it is a property of the implementation, not of the instruction set — the program cannot tell whether it runs on a pipeline, it only finishes sooner.

The table is the exam's way of writing Fig. 2: rows are instructions, columns are clock cycles C1, C2 …, and each cell names the stage. (The notes' Table 1 on p. 15 is the transpose, rows for cycles and columns for stages; same diagonal.)

One honesty note. This table assumes what the notes assume for the ideal case: that no instruction needs a value a previous one has not yet produced, and that the branch does not disturb the fetching. Our loop breaks both — FADD needs the F2 that LD produces, and BNE jumps. Lesson 2 and lesson 4 charge for that. Today the pipeline is ideal.

The formula behind the 18 ns

Count cycles in the table. The first instruction needs five; every later one finishes one cycle after its predecessor. Write \(k\) for the number of stages, \(n\) for the number of instructions and \(T\) for the stage time (which is the clock period):

1. Without a pipeline every instruction takes all \(k\) stage times, one after another:

   \[ T_{\text{seq}} = n \, k \, T . \]

2. On the pipeline the first instruction finishes in cycle \(k\). Instruction number \(i\) (counting from 1) enters one cycle later than its predecessor and therefore finishes one cycle later: in cycle \(k + (i - 1)\).

3. The last instruction has \(i = n\), so the whole program ends in cycle \(k + n - 1\):

   \[ T_{\text{pipe}} = (k + n - 1) \, T . \]

   With \(k = 5\), \(n = 5\), \(T = 2\) ns: \(9 \times 2 = 18\) ns, the number of the hook. The \(k - 1 = 4\) extra cycles are the fill: the cycles before the first instruction comes out, during which the stops behind it are still filling up.

4. The speedup is the ratio; the stage time cancels:

   \[ S(n) = \frac{n \, k \, T}{(k + n - 1) \, T} = \frac{n \, k}{k + n - 1} . \]

5. Divide numerator and denominator by \(n\). As \(n\) grows, \((k - 1) / n\) vanishes and the speedup climbs towards \(k\) — never reaching it:

   \[ S(n) = \frac{k}{1 + (k - 1)/n} \;\longrightarrow\; k . \]

   For five instructions: \(25 / 9 = 2.78\). For a hundred: \(500 / 104 = 4.81\).

Pipelining raises throughput, not speed: one instruction still takes \(k\) cycles, but once the line is full one completes every cycle, and \(n\) instructions take \(k + n - 1\) cycles.

What each instruction does at each stop

All five instructions march through the same five stages, but they do different things there — and in some stages nothing at all. The notes give one small table per instruction class (pp. 16–17); here they are side by side, in the loop's own instructions. The cell marked "born" is where the instruction's result first exists; the cell marked "idle" is a stage the instruction passes through without doing anything.

Three facts in this table are worth more than the rest, because the next lessons are built on them. A load's value is born at the end of MEM, one stage later than an ALU result, because the load needs the ALU for the address and then the memory: this is why a load's consumer will cost a stall that no wire can remove. A store reads two registers in ID — the base and the data — although the data is needed only in MEM: ID is the only stage with read ports, so the value is read early and carried along. And a branch compares in EX and loads the PC in MEM; the instructions fetched meanwhile were fetched blind. That is the control hazard the notes name on p. 17 and lesson 4 is about.

Why an idle stage is not a skipped stage

Look at the grey cells and ask the obvious question: the FADD has its result at the end of EX, so why not write it in the very next cycle and let the instruction leave the pipeline early? Because of the fifth column of Fig. 1. The register file has one write port, and in any cycle the instruction one step ahead may be using it. If FADD jumped from EX to WB, it would reach WB in cycle 5, the same cycle as the LD in front of it — two writes, one port. Worse, an ALU instruction that follows a load would then write its register before the load writes its own, and results would land in the register file out of program order. Keeping every instruction in the pipe for exactly five cycles guarantees that at most one instruction is in WB per cycle and that writes happen in order. The idle stage is the price of that guarantee, and the price is zero: the stage was going to exist anyway.

One last look at the table before moving on. F2 is born at the end of cycle 4, in the LD's MEM stage — but FADD reads F2 in its ID stage, which is cycle 3. The ideal table of the previous section quietly reads a value that does not exist yet. That is the data hazard, and what the hardware does about it is the whole of lesson 2.

An ALU result is born at the end of EX, a loaded value at the end of MEM; an instruction with nothing to do in a stage still spends the cycle there.

What travels between the stops: the pipeline registers

Between any two stations of a car factory there is a short stretch of conveyor that holds the car — and its work order — while the next station is busy. The pipeline has the same thing between any two stages: a pipeline register, a bank of flip-flops that is written at the end of every clock cycle and read in the next. There are four of them, named after the stages they separate: IF/ID, ID/EX, EX/MEM and MEM/WB (notes, p. 19 and Fig. 4 on p. 18). All four are clocked together, so on every clock edge the whole pipeline advances by one stop.

What must a register hold? Everything the instruction inside it still needs for the stages ahead — because the instruction word itself stays behind in IF/ID and is overwritten by the next fetch one cycle later. The notes list the contents in one line each; here is the full list, with the control-signal names of Patterson and Hennessy, which the notes do not spell out:

• IF/ID: the instruction word (32 bits) and PC + 4.
• ID/EX: PC + 4; the two values read from the register file; the sign-extended immediate; the number of the destination register rd; and the control signals decoded from the opcode for the three stages ahead — for EX (ALUOp, ALUSrc: register or immediate as the second operand), for MEM (MemRead, MemWrite, Branch) and for WB (RegWrite, MemtoReg: write the ALU result or the loaded word).
• EX/MEM: the ALU result, which for a load or store is the address; the value to be stored; for a branch the target address and the outcome of the comparison; rd; the control signals for MEM and WB.
• MEM/WB: the word read from memory, or the ALU result passed through; rd; the control signals for WB.

Two things travel in every register and are easy to forget at the exam: the destination register number and the control bits. Decoding happens once, in ID; from then on the instruction carries its own work order, and each stage consumes the part addressed to it, so the control word gets shorter from register to register. The write-back stage, three cycles after decode, learns which register to write from the rd field of MEM/WB, not from the instruction word, which by then belongs to an instruction three places behind.

Why have the registers at all? Not for the reason the notes give on p. 19 ("they eliminate the need for re-fetching or re-decoding instructions at each cycle" — re-fetching was never on the table). The real reason is the clock. Without a register between two stages, their logic would form one long combinational path, and the clock period would have to cover both; the register cuts the path, so that the period only has to cover one stage. That is what lets the clock run at the pace of a stage instead of the pace of an instruction, which is the entire point of the technique. The registers are also where the forwarding hardware of the next lesson reads its values from: a result that sits in EX/MEM can be handed to the ALU without waiting for WB.

The instruction word stays in IF/ID for one cycle; everything the later stages need — operands, immediate, destination number, control bits — travels with the instruction through ID/EX, EX/MEM and MEM/WB.

The puzzle, solved — and what spoils the ideal line

Back to the hook. Five instructions, 50 ns sequential, 18 ns pipelined, and the notes' "5×". You now have the subtraction: the pipeline needs \(k + n - 1 = 9\) cycles, not \(n = 5\), because the first instruction must travel the whole line before anything comes out — four fill cycles, \(k - 1\). The speedup is \(nk / (k + n - 1) = 25/9 = 2.78\). You get the rest back only by amortising the fill: 100 instructions give 4.81, 1000 give 4.98, and the 5 is a ceiling, approached and never reached. Add the register overhead and the ceiling itself drops below 5.

The same four cycles appear on the example A4 formula sheet the course page shows as "Clock Cycles = IC + Stall Cycles + 4". The "+4" is this fill, counted once per program; the "stall cycles" are the price of what this lesson assumed away. Lesson 3 turns these into CPI and MIPS.

Three things spoil the ideal line, and the notes name them on pp. 10, 17 and 20–21. A structural hazard: two instructions need the same resource in the same cycle — with one memory for instructions and data, the IF of one instruction and the MEM of another would collide, which is why the datapath has two. A data hazard: an instruction needs a value an earlier one has not produced yet — our FADD and its F2. A control hazard: the next PC is not known when the next fetch should start — our BNE. Each of them costs cycles, each has a fix, and each fix is a lesson: data hazards and forwarding next, branches in lesson 4, the hazards that return with multi-cycle units in lesson 7.

One rule to carry with you until lesson 7, where exceptions are treated: whatever overlapping the pipeline does, an exception must look as if the faulting instruction had executed alone — everything before it completed, nothing after it started. Overlap is allowed to change timing, never meaning.