Pipelining in Embedded Systems

What is Pipelining?

Pipelining is a technique used in processor design to increase instruction throughput. Instead of completing one instruction entirely before starting the next, a pipelined processor breaks each instruction into multiple stages and works on several instructions simultaneously, each at a different stage.Think of it like a car assembly line. You do not build one car completely before starting the next. Instead, while one car gets its engine installed, the next car is getting its body painted, and another is getting its wheels attached. Each station works in parallel on different cars.Similarly, a pipelined processor works on different stages of different instructions at the same time.

Instruction Execution Without Pipelining

Without pipelining, a processor completes every stage of one instruction before starting the next:

Time:      1   2   3   4   5   6   7   8   9  10  11  12

Instr 1:  [F] [D] [E] [W]
Instr 2:                   [F] [D] [E] [W]
Instr 3:                                    [F] [D] [E] [W]

F = Fetch, D = Decode, E = Execute, W = Write Back

3 instructions take 12 clock cycles.

Each instruction takes 4 cycles, and 3 instructions take 12 cycles total.

Instruction Execution With Pipelining

With pipelining, the next instruction starts as soon as the first stage is free:

Time:      1   2   3   4   5   6

Instr 1:  [F] [D] [E] [W]
Instr 2:      [F] [D] [E] [W]
Instr 3:          [F] [D] [E] [W]

3 instructions take 6 clock cycles (instead of 12).

After the pipeline is full, one instruction completes every clock cycle. This effectively doubles the throughput in this 4-stage example.

Pipeline Stages

Different processors have different numbers of pipeline stages. Here are common examples:

ARM7TDMI (3-Stage Pipeline)

The classic ARM7 used in many older embedded systems has a simple 3-stage pipeline:

1. Fetch: Read the instruction from memory
2. Decode: Interpret the instruction and read registers
3. Execute: Perform the operation and write results

ARM Cortex-M3/M4 (3-Stage Pipeline)

Modern Cortex-M processors also use a 3-stage pipeline but with branch speculation:

1. Fetch: Fetch instruction from flash or cache
2. Decode: Decode instruction, predict branches
3. Execute: Execute the instruction, access memory, write back

ARM Cortex-A (Deeper Pipelines)

Application processors used in systems like Raspberry Pi have deeper pipelines (8-13+ stages) for higher clock speeds:

1. Fetch 1
2. Fetch 2
3. Decode
4. Issue
5. Execute 1
6. Execute 2
7. Memory access
8. Write back

General rule: Deeper pipelines allow higher clock speeds but have greater penalties when the pipeline stalls.

Pipeline Hazards

Pipelining does not always run smoothly. Situations that prevent the next instruction from executing in its designated stage are called hazards. There are three types:

1. Data Hazard

Occurs when an instruction depends on the result of a previous instruction that has not completed yet.

ADD R1, R2, R3    // R1 = R2 + R3
SUB R4, R1, R5    // R4 = R1 - R5  (needs R1 from previous instruction!)

Timeline without forwarding:
ADD:  [F] [D] [E] [W]          -- R1 written in stage 4
SUB:      [F] [D] [STALL] [E] [W]  -- must wait for R1

Solutions:

• Data forwarding (bypassing): Hardware passes the result directly from the Execute stage to the next instruction without waiting for Write Back.
• Pipeline stall (bubble): Insert a wait cycle until the data is available.
• Compiler reordering: The compiler rearranges instructions to put unrelated instructions between dependent ones.

2. Control Hazard (Branch Hazard)

Occurs with branch instructions (if/else, loops, function calls). The processor does not know which instruction to fetch next until the branch condition is evaluated.

CMP R1, #10        // Compare R1 with 10
BEQ label          // Branch if equal
ADD R2, R3, R4     // This might or might not execute
...
label:
SUB R5, R6, R7     // Branch target

When the processor encounters the branch, it has already started fetching the next instruction. If the branch is taken, those fetched instructions must be discarded, which is called a pipeline flush.Solutions:

• Branch prediction: The processor guesses whether the branch will be taken. If the guess is correct, there is no penalty. If wrong, the pipeline is flushed.
• Delayed branching: Some architectures (like early MIPS) always execute the instruction after the branch, giving the pipeline time to resolve the branch.
• Branch target buffer: Hardware that remembers where previous branches went.

On ARM Cortex-M3 (3-stage pipeline): A taken branch causes a penalty of 1-3 cycles because 1-2 instructions in the pipeline must be discarded.

3. Structural Hazard

Occurs when two instructions need the same hardware resource at the same time. For example, if there is only one memory port and both the Fetch and Memory Access stages need it simultaneously.Solutions:

• Separate instruction and data caches (Harvard architecture, used in most ARM processors)
• Adding duplicate hardware resources

Why Pipelining Matters for Embedded Developers

You might think pipelining is purely a hardware concern. But it affects how you write software:

1. Branch-Heavy Code Has Penalties

// Many branches = many potential pipeline flushes
for (int i = 0; i < 100; i++) {
    if (data[i] > threshold) {    // branch
        count++;
    }
}

// Branchless alternative (faster on pipelined processors)
for (int i = 0; i < 100; i++) {
    count += (data[i] > threshold);  // no branch
}

2. Loop Unrolling

Reducing the number of loop iterations reduces branch penalties:

// Original - branch check every iteration
for (int i = 0; i < 100; i++) {
    result += data[i];
}

// Unrolled - branch check every 4 iterations
for (int i = 0; i < 100; i += 4) {
    result += data[i];
    result += data[i+1];
    result += data[i+2];
    result += data[i+3];
}

3. Interrupt Latency

When an interrupt occurs, the pipeline must be flushed (partially or fully) to start executing the interrupt handler. Deeper pipelines mean higher interrupt latency. This is one reason why Cortex-M processors have shorter pipelines than Cortex-A: they prioritize low interrupt latency for real-time applications.

4. Deterministic Timing

In real-time systems, you need to know the worst-case execution time. Pipeline stalls and branch mispredictions make timing less predictable. Simple pipelines (like Cortex-M) are more deterministic than deep pipelines with complex branch prediction (like Cortex-A).

Pipeline Depth: Trade-offs

Shallow Pipeline (3-5 stages)	Deep Pipeline (10+ stages)
Lower clock speed possible	Higher clock speed possible
Small branch penalty	Large branch penalty
More deterministic timing	Less deterministic timing
Lower interrupt latency	Higher interrupt latency
Simpler hardware, less power	Complex hardware, more power
Used in: Cortex-M, AVR, MSP430	Used in: Cortex-A, x86, MIPS

Summary

Pipelining is a fundamental processor technique that increases throughput by overlapping the execution of multiple instructions:

• Without pipelining, the CPU processes one instruction at a time from start to finish
• With pipelining, different stages of different instructions are processed simultaneously
• Pipeline hazards (data, control, structural) can cause stalls that reduce performance
• Embedded microcontrollers (Cortex-M) use short pipelines for determinism and low latency
• Application processors (Cortex-A) use deep pipelines for high clock speeds
• Writing branch-friendly code and understanding pipeline behavior helps you write faster embedded software

For embedded developers, the key takeaway is: know your processor’s pipeline depth, understand the cost of branches, and consider these factors when writing time-critical code.

Practical Impact: What Pipelining Means for Your ISR Code

If you write timing-critical interrupt service routines, pipelining directly affects how your code behaves. Here are the practical implications that matter when you are writing real firmware.

1. Interrupt Latency Is Not Just “Cycles to Enter ISR”

When an interrupt fires, the processor must:

• Flush the pipeline — all partially-executed instructions in the pipe are discarded. On a 3-stage pipeline (ARM Cortex-M0), this wastes up to 2 instructions. On a 14-stage pipeline (Cortex-A8), up to 13 instructions are thrown away.
• Save context — push registers onto the stack. ARM Cortex-M does this automatically (stacking 8 registers), adding 12 cycles on Cortex-M3.
• Refill the pipeline — the first instruction of your ISR does not execute immediately. The pipeline must fill again (3 cycles on Cortex-M0).

Total worst-case latency on Cortex-M3: 12 cycles (stacking) + 6 cycles (pipeline flush + refill) = 18 cycles minimum. On Cortex-M0, it is 16 cycles. On deeper pipelines (Cortex-A series), it can exceed 40 cycles. This matters when you are trying to respond to a signal within microseconds.

2. Branch Instructions Inside ISRs Are Expensive

Every if-else or switch inside your ISR causes a branch. On a pipelined processor, a mispredicted branch flushes the pipeline again. In a tight ISR, this can double your execution time.

/* Slow: branch-heavy ISR on pipelined processor */
void TIMER1_IRQHandler(void)
{
    if (mode == MODE_PWM) {
        update_pwm();        /* Branch 1 */
    } else if (mode == MODE_CAPTURE) {
        read_capture();      /* Branch 2 */
    } else {
        handle_overflow();   /* Branch 3 */
    }
}

/* Faster: use a function pointer — one indirect call, no branches */
static void (*timer_handler)(void) = update_pwm;

void TIMER1_IRQHandler(void)
{
    timer_handler();  /* Single indirect call — pipeline-friendly */
}

/* Change handler in main code, not in the ISR */
void set_timer_mode(uint8_t mode)
{
    if (mode == MODE_PWM)     timer_handler = update_pwm;
    if (mode == MODE_CAPTURE) timer_handler = read_capture;
}

3. Loop Unrolling Matters More Than You Think

A loop like for (i = 0; i < 4; i++) creates a branch at every iteration (the loop-back jump). On a 3-stage pipeline, each iteration wastes 2 cycles on the branch. For a 4-iteration loop, that is 8 wasted cycles. If the loop body is only 3 instructions, the branch overhead is 40% of total execution time.

/* With loop: 4 branches × 2 wasted cycles = 8 extra cycles */
for (int i = 0; i < 4; i++) {
    buffer[i] = SPI_read();
}

/* Unrolled: zero branch overhead */
buffer[0] = SPI_read();
buffer[1] = SPI_read();
buffer[2] = SPI_read();
buffer[3] = SPI_read();

Rule of thumb: If your ISR has a small, fixed-iteration loop (2–8 iterations), unroll it manually. For larger loops, let the compiler handle it with -funroll-loops. The tradeoff is code size vs execution speed — in an ISR, speed usually wins.