Buffer Overflow Attacks Explained: The Classic Memory Bug, Demonstrated Safely

Quick Answer

A buffer overflow happens when a program writes more data into a fixed-size block of memory than that block can hold, and the extra bytes spill into whatever memory sits right after it — corrupting a neighboring variable, or in the worst case, a function's own return address. We demonstrated the core mechanism safely with a small Python script that deliberately overflows an 8-byte buffer into an adjacent integer, flipping it from 0 to 1 without ever assigning to it directly — the same effect a real stack-smashing exploit relies on in unsafe C code, shown here without any compiled exploit or working attack.

What a Buffer Overflow Actually Is

A buffer is just a fixed-size, contiguous block of memory reserved to hold data — a char buffer[8] in C reserves exactly 8 bytes, no more. The language itself does not check, at the point where you write into that buffer, whether what you're writing actually fits. If a function writes 20 bytes into an 8-byte buffer using an operation that doesn't check bounds, those extra 12 bytes don't vanish or get rejected — they simply get written into whatever memory happens to sit immediately after the buffer, overwriting whatever was stored there, whether that's another variable, a saved register, or a function's return address.

This is specifically a consequence of how C and C++ (and other languages that compile directly to raw memory operations without automatic bounds checking) manage memory, and it's one of the oldest named vulnerability classes in computer security — buffer overflow vulnerabilities were documented as early as 1972 in a US Air Force computer security study, and the technique was popularized for a mainstream security audience by Aleph One's widely read 1996 paper 'Smashing The Stack For Fun And Profit,' which walked through exactly how an attacker could use a stack-based buffer overflow to hijack a program's execution. Nearly three decades later, buffer overflows are still regularly found in real, shipped software — not because the fix is unknown, but because C and C++ code that manually manages buffer sizes keeps getting written, and manual bounds-checking is the kind of thing that's reliably correct 999 times and silently wrong the one time that matters.

The classic, most dangerous version of this bug happens on the call stack — the region of memory where a running function keeps its local variables, and critically, the address execution should return to once that function finishes. Local variables (including fixed-size buffers) and the return address are laid out close together in that same stack frame, which is exactly why an overflowing write into a local buffer has a path to reach something as consequential as the return address itself, not just another harmless local variable.

How the Classic Stack-Smashing Exploit Works

The textbook vulnerable pattern looks almost exactly like this:

void vulnerable_function(char *user_input) {
    char buffer[8];
    strcpy(buffer, user_input);  // strcpy NEVER checks the destination's size
}

strcpy copies bytes from its source until it hits a null terminator — it has no idea how big buffer actually is, and no way to find out from its own arguments. If user_input is longer than 8 bytes, strcpy keeps writing straight past the end of buffer, into whatever memory sits immediately after it on the stack — which, depending on exact layout and compiler, can include other local variables, saved register values the function needs to restore before returning, and the saved return address that tells the CPU where to resume execution once vulnerable_function finishes.

In the simplest version of the classic attack, an attacker crafts input specifically sized and shaped so that the overflow doesn't just corrupt the return address randomly — it overwrites it with the address of code the attacker wants to run instead, often code the attacker has also smuggled in as part of the same oversized input (historically padded with 'NOP sled' filler bytes to make landing on the right instruction more forgiving). When the function returns, instead of resuming the caller's legitimate code, the CPU jumps to the attacker's address — at which point the attacker's injected instructions execute with whatever privileges the vulnerable program had. That's the full chain: unchecked write, past the buffer, into the return address, redirecting execution. This article demonstrates only the first link of that chain — unchecked write corrupting adjacent memory — safely and without ever assembling a working exploit.

It's worth knowing that stack-based overflows (the kind described above) are only one variant. A heap overflow follows the same core idea — unchecked write, spilling past the end of an allocation — but the allocation lives in the heap instead of the stack, so instead of corrupting a return address, it typically corrupts heap metadata (the bookkeeping a memory allocator keeps about allocated and free blocks) or an adjacent object's fields, which attackers have historically turned into code execution through different, more involved techniques than a direct return-address overwrite. An off-by-one overflow is the same bug in miniature: writing exactly one byte past the end of a buffer, often from a loop boundary condition that should have been < instead of <=, which sounds trivial but has been enough on its own to corrupt a critical adjacent byte (like the low byte of a saved pointer) in several well-documented real-world vulnerabilities. The common thread across all of these variants, and the reason this article treats the stack-based version as the representative case, is always the same: a write operation that trusts a length it never independently checked.

A Safe Demonstration: Flipping a Flag Without Touching It

There's no C compiler available in the environment this article was written in, so rather than describe the mechanism only in prose, here's a real, executed demonstration of the exact underlying effect — unbounded writes past a buffer corrupting an adjacent variable — built in Python using ctypes, which exposes raw memory the same way C does, without needing to compile and run an actual exploit against anything. This is a controlled teaching example, not a working attack against any real program.

import ctypes

class StackFrame(ctypes.Structure):
    _fields_ = [
        ("buffer", ctypes.c_char * 8),   # like: char buffer[8];
        ("is_admin", ctypes.c_int32),    # like: int is_admin = 0;  laid out right after buffer
    ]

frame = StackFrame()
frame.buffer = b"hello"
frame.is_admin = 0
print("Before:", frame.buffer, frame.is_admin)

# The unsafe C equivalent being simulated here: strcpy(buffer, user_input);
user_input = b"AAAAAAAA" + b"\x01\x00\x00\x00"  # exactly 12 bytes: fills buffer + flips is_admin
ctypes.memmove(ctypes.addressof(frame), user_input, len(user_input))

print("After:", frame.buffer, frame.is_admin)

Real output from running exactly this:

sizeof(StackFrame) = 12 bytes  (8-byte buffer + 4-byte int, no padding)
Before overflow:  buffer = b'hello'  is_admin = 0
After overflow:   buffer = b'AAAAAAAA'  is_admin = 1  <- flipped without ever assigning to is_admin

is_admin changes from 0 to 1 even though no line of this program ever assigns to is_admin directly — it's flipped purely as a side effect of writing past the end of buffer, into the exact bytes of memory the struct's layout happens to place right after it. This is precisely the mechanism a real stack-smashing exploit relies on, just demonstrated against a harmless integer flag instead of a function's return address, and using a deliberately bounded, 12-byte write that stays inside memory this script itself allocated — not an uncontrolled overflow into arbitrary process memory.

Two details in that output are worth sitting with. First, the 20-byte version of this same script (tried before settling on the 12-byte one shown above) wrote past the end of the entire allocated structure, not just past buffer — which is a reminder that an overflow's blast radius is bounded only by how much extra data gets written, not by any property of the buffer itself. Second, the corruption here is silent: nothing printed an error, nothing crashed, and is_admin simply holds a different value than any line of code set it to. That silence is exactly why this bug class is dangerous in real programs — a corrupted flag or pointer can sit unnoticed for many more lines of execution before its effect becomes visible, if it ever becomes visible at all before an attacker relies on it.

The Real Defenses: Canaries, ASLR, and DEP/NX

Modern systems don't rely on programmers simply writing bounds checks correctly every time — they add layers of defense specifically designed to catch or block this exact attack chain, each closing off a different step of it. Stack canaries work by placing a known, unpredictable value — the canary — in memory between a function's local buffers and its saved return address. Before the function returns, compiler-inserted code checks that the canary still has its original value; if a buffer overflow has overwritten the return address, it necessarily overwrote the canary sitting in front of it too, and the mismatch is caught before the corrupted return address is ever used, crashing the program safely instead of letting it jump to attacker-controlled code.

Data Execution Prevention (DEP), built on a hardware feature called the NX bit on AMD processors (and XD on Intel), marks memory pages that are meant to hold only data — like the stack, where a classic overflow's injected shellcode would otherwise sit — as non-executable at the hardware level. Even if an attacker successfully overwrites a return address to point at injected code sitting on the stack, the CPU itself refuses to execute instructions from that page, because the NX/XD bit says it isn't allowed to.

Address Space Layout Randomization (ASLR) attacks a different step: it randomizes where key regions of a program's memory — the stack, the heap, loaded libraries — actually sit in the process's address space on each run. Even an attacker who successfully corrupts a return address needs to know a specific, valid address to redirect it to; ASLR means that address is different and unpredictable every time the program runs, making a hardcoded target address from a previous analysis useless against a freshly started process.

None of these three is individually unbeatable, and that's deliberate — stack canaries can be defeated if a separate information-disclosure bug lets an attacker read the canary's value before crafting the overflow; ASLR can be weakened by bugs that leak memory addresses; DEP/NX can be worked around using techniques that chain together existing executable code instead of injecting new code (return-oriented programming). The actual security property these mitigations provide together is cumulative difficulty: a successful modern exploit against a program with all three enabled typically has to find a buffer overflow and defeat ASLR and bypass DEP, rather than just the single bug that was often enough to get code execution in the 1990s landscape that Aleph One's original paper described.

Why Modern Languages Mostly Sidestep This Bug Class

The deepest fix isn't a runtime mitigation at all — it's not giving the bug a way to exist in the first place. Languages like Rust, Go, and Java perform bounds checking on array and buffer accesses automatically: writing or reading past the declared end of an array raises an immediate, well-defined error (a panic in Rust, an exception in Java, a runtime error in Go) rather than silently continuing to write into adjacent memory. The performance cost of that check is small and, in Rust's case, the compiler can often prove a bounds check is unnecessary at compile time and eliminate it — these languages get memory safety against this specific bug class essentially for free from a correctness standpoint, by design, not through a mitigation bolted on afterward.

This is also why 'use a memory-safe language instead of C/C++' shows up so often as the top recommendation in modern security guidance for new projects — not as language-preference advice, but because buffer overflows and the broader category of memory-corruption bugs they belong to have been, for decades, among the most consequential and most frequently exploited vulnerability classes in real shipped software, and languages that check bounds automatically remove the bug class rather than defending against its consequences after the fact.

For teams that are stuck with C or C++ for legitimate reasons — existing codebases, hardware-level work, performance-critical paths — practical defenses exist below the language level: use length-bounded string functions (strncpy, snprintf) instead of their unbounded equivalents (strcpy, sprintf) everywhere user-controlled or externally sourced data touches a fixed buffer; compile with stack-protector flags and keep ASLR and DEP/NX enabled at the OS level rather than disabling them for convenience; and treat any function that copies data into a fixed-size buffer based on a length it doesn't independently verify as a standing candidate for a security review, because that's exactly the shape of code every real-world buffer overflow in the historical record has taken.

Frequently Asked Questions