← Systems Laboratory

Lab: The Attack Lab, Buffer Overflow Exploits

Published on 2026-09-12·v1.0

Learning Objectives

  • Construct a code-injection buffer overflow attack that overwrites a saved return address on the stack with the address of hand-crafted shellcode.
  • Explain why a defense that prevents executing code on the stack (a non-executable stack) defeats code injection specifically, and why that defense alone does not close the underlying vulnerability.
  • Construct a return-oriented-programming (ROP) attack that chains together short, existing instruction sequences ("gadgets") already present in the target binary, without injecting any new code at all.
  • Explain, precisely, the stack-layout knowledge, exactly where the return address sits relative to a buffer, both attacks depend on.

Context & Motivation

Stack Smashing and Buffer Overflows and The x86-64 Runtime Stack: Call and Ret already proved, on paper, that an unchecked write past a fixed-size stack buffer can overwrite adjacent stack memory, including a saved return address, and that overwriting a return address redirects control flow to wherever the attacker chooses the moment the current function returns. This lab is where that proof becomes a real, working exploit against a real, compiled binary, CMU's own Attack Lab, escalating across five levels from a straightforward code-injection attack to harder return-oriented-programming attacks that work even once code injection itself is defended against.

Core Theory

Nothing about why a stack buffer overflow can overwrite a return address is re-derived here; that argument belongs to stack-smashing-and-buffer-overflows and the-x86-64-runtime-stack-call-and-ret. This lab is the discipline of turning that argument into working exploit bytes against a specific, real stack layout, and of recognizing why the strongest of the two attack families here, ROP, still succeeds after the simpler one is defended against.

Worked Examples

The stack layout every attack in this lab depends on

text
Higher addresses +-------------------------+ | saved return address | <- the exact target this whole lab manipulates +-------------------------+ | saved base pointer | +-------------------------+ | local buffer (e.g. 32B) | <- the vulnerable function's own fixed-size array +-------------------------+ Lower addresses, stack grows downward

Every attack in this lab depends on the same underlying fact, already established theoretically: the return address sits at a fixed, known offset above a vulnerable buffer, so a write that overflows that buffer by exactly the right number of bytes lands precisely on the return address, not somewhere else on the stack.

Level 1-3: code injection

text
Attacker-supplied input, exactly matching the buffer's overflow offset: [32 bytes of buffer padding] [8 bytes: new "return address" pointing INTO the attacker's own input, specifically at the shellcode bytes below] [shellcode: raw x86-64 machine code the attacker wrote, e.g. instructions that call a specific function with a chosen argument]

The overwritten return address points not to any existing function but directly back into the attacker's own supplied input, at bytes that are themselves valid x86-64 instructions; when the vulnerable function returns, the processor begins executing those attacker-supplied bytes as code.

Level 4-5: return-oriented programming, once code injection is defended against

text
A non-executable stack (a real, standard defense) makes the shellcode above unexecutable: the processor refuses to run instructions fetched from stack memory at all, regardless of what bytes are there. ROP's answer: instead of injecting NEW code, chain together addresses of small, existing instruction sequences ("gadgets") ALREADY present in the target binary's own executable code, each one ending in a `ret` instruction that pops the NEXT gadget's address off the stack and jumps to it: [32 bytes of buffer padding] [address of gadget 1: e.g. "pop %rdi; ret" — pops a value into %rdi] [the value to pop into %rdi: the attacker's chosen argument] [address of gadget 2: e.g. the address of a real function to call]

Each gadget's own ret is what makes the chain work: it pops the next address in the attacker's crafted stack data and jumps there, so control flow hops from one small, pre-existing snippet of the binary's own code to the next, executing attacker-chosen logic without a single new instruction ever being injected.

Step 1 — finding usable gadgets in the target binary

text
$ objdump -d ctarget | grep -B 1 'ret$' 4019a2: 5f pop %rdi 4019a3: c3 ret 4019c3: 58 pop %rax 4019c4: c3 ret

Real ROP attacks are built from exactly this kind of search: scanning a binary's own compiled code for short instruction sequences, often unintended ones that happen to occur inside longer, legitimate instructions, ending in ret, then chaining their addresses together to perform a chosen sequence of operations.

Common Misconceptions & Pitfalls

  • "A non-executable stack fully closes the buffer overflow vulnerability." It defeats code injection specifically (Levels 1-3), by refusing to execute newly injected bytes, but the underlying vulnerability, an attacker's ability to overwrite the return address and control what executes next, is unchanged; ROP (Levels 4-5) shows that vulnerability is still fully exploitable using only the target's own already-executable code.
  • "ROP requires writing new malicious code, just structured differently." It requires the opposite: no new instructions are ever introduced; every instruction executed during a ROP attack was already present, and already marked executable, in the target binary before the attack began, which is precisely what lets it bypass a non-executable-stack defense.
  • "The exact buffer-overflow offset can be guessed by trial and error without understanding the stack layout." The offset that lands precisely on the return address is a specific number of bytes, determined by the vulnerable function's actual stack frame layout; getting it wrong overwrites the wrong memory (often crashing the program instead of redirecting it), which is why the-x86-64-runtime-stack-call-and-ret's stack-frame model is a real prerequisite, not optional background, for this lab.

Summary

This lab turns stack-smashing-and-buffer-overflows's theoretical proof into a real, working exploit against CMU's own Attack Lab: three code-injection attacks that overwrite a return address to redirect execution into attacker-supplied shellcode, followed by two harder return-oriented-programming attacks that achieve the same control-flow hijacking using only gadgets, short instruction sequences ending in ret, already present in the target's own compiled code. The escalation from code injection to ROP demonstrates directly why a non-executable stack, a real, standard defense, closes one specific attack vector without closing the underlying vulnerability, exactly the distinction the-x86-64-runtime-stack-call-and-ret's stack-frame model makes precise.

Documentation Links