Understanding memory in C++

Memory in C++ becomes significantly easier once you stop thinking about pointers as mysterious syntax and start visualizing how the operating system and CPU actually structure physical and virtual memory.

At the hardware boundary, every variable, struct, and dynamically allocated buffer is just a sequence of bytes residing at an integral memory address.

The Virtual Memory Layout of a Process

When your operating system executes an ELF binary on Linux or a Mach-O binary on macOS, it does not give the program raw access to physical RAM. Instead, the kernel and the CPU’s Memory Management Unit (MMU) construct a Virtual Address Space.

A typical 64-bit user space process layout is organized into distinct functional segments:

High Addresses (0x7FFF_FFFF_FFFF)
+-------------------------------------------------------+
| Kernel Space (mapped for syscalls, inaccessible in U) |
+-------------------------------------------------------+
| Stack (grows downward toward lower addresses)         |
|   |                                                   |
|   v                                                   |
|                                                       |
|   ^                                                   |
|   |                                                   |
| Heap (grows upward via brk / sbrk / mmap)             |
+-------------------------------------------------------+
| BSS Segment (uninitialized global/static variables)   |
+-------------------------------------------------------+
| Data Segment (initialized global/static variables)    |
+-------------------------------------------------------+
| Text Segment (compiled machine code instructions)     |
+-------------------------------------------------------+
Low Addresses (0x0000_0000_0000)

Understanding where your data lives within these regions is the difference between writing efficient, crash-free systems code and fighting segmentation faults.

Stack Allocation: Deterministic and Fast

The stack is governed directly by CPU architecture registers (typically the stack pointer rsp and base frame pointer rbp on x86_64).

When a function executes:

The caller pushes parameters and the return instruction address onto the stack.
The stack pointer moves downward to reserve space for local variables.
When the function returns, the stack pointer increments back up to its prior position.

#include <iostream>

void demonstrateStack() {
    int age = 24;              // 4 bytes allocated on the stack
    double coordinate = 37.77; // 8 bytes allocated on the stack
    char buffer[64];           // 64 contiguous bytes allocated on the stack

    std::cout << "Local int address: " << &age << "\n";
    std::cout << "Local double address: " << &coordinate << "\n";
} // All 76 bytes are instantly released as the stack pointer resets

Because allocating on the stack only requires a single CPU arithmetic instruction (sub rsp, N), allocation takes sub-nanosecond time. Furthermore, because stack data is continuously recycled within a narrow address range, it exhibits outstanding CPU L1/L2 cache locality.

Heap Allocation: Dynamic Lifetimes

Unlike stack variables whose lifetime is tied to their lexical scope, heap allocations persist until they are explicitly released.

When you call new or malloc, execution branches into a user-space memory allocator (such as glibc ptmalloc, Google’s tcmalloc, or jemalloc):

#include <iostream>
#include <memory>

struct SensorReading {
    uint64_t timestamp;
    double value;
};

void dynamicAllocationExample() {
    // 1. Primitive heap allocation
    SensorReading* rawPtr = new SensorReading{1726000000, 42.195};

    std::cout << "Sensor value: " << rawPtr->value << "\n";
    std::cout << "Heap address: " << rawPtr << "\n";

    // Danger: If we return here without calling delete, memory leaks forever!
    delete rawPtr;

    // 2. Modern idiomatic C++: RAII and Smart Pointers
    auto safePtr = std::make_unique<SensorReading>(SensorReading{1726000001, 84.39});
    std::cout << "Safe sensor value: " << safePtr->value << "\n";
    // Deallocation happens automatically when safePtr leaves this scope
}

The heap carries distinct trade-offs:

Allocation latency: The allocator must search free lists or memory arenas to find a contiguous block of sufficient size.
Fragmentation: Alternating allocations and deallocations can leave small gaps of unmapped memory that cannot satisfy larger future allocations.
Metadata overhead: Every allocation typically records bookkeeping headers (usually 8–16 hidden bytes preceding the allocated address).

Stack vs. Heap: Architectural Trade-offs

Dimension	Stack Memory	Heap Memory
Allocation Speed	Immediate ( $O(1)$ stack pointer decrement)	Slower (allocator search, syscalls like `mmap` or `brk`)
Deallocation	Automatic upon exiting scope	Manual or RAII-driven via smart pointers
Size Limitation	Constrained (typically 8MB per thread)	Bound only by virtual memory and swap limits
Cache Locality	Extremely high (hot in L1/L2 caches)	Variable (can cause cache line misses if fragmented)
Safety Hazards	Stack overflow on deep recursion	Memory leaks, dangling pointers, double frees

Pointers and Memory Addresses Under the Microscope

A pointer is simply an unsigned integer whose value represents a memory address. On 64-bit systems, all pointers are 8 bytes in size regardless of the underlying data type they reference:

#include <iostream>

void inspectPointers() {
    int value = 42;
    int* ptr = &value;

    std::cout << "Value:            " << value << "\n";
    std::cout << "Address of value: " << &value << "\n";
    std::cout << "Contents of ptr:  " << ptr << "\n";
    std::cout << "Dereferenced ptr: " << *ptr << "\n";
    std::cout << "Size of ptr:      " << sizeof(ptr) << " bytes\n";
    std::cout << "Size of int:      " << sizeof(value) << " bytes\n";
}

Visual representation of what lives in memory:

Variable:       value                   ptr
Type:           int (4 bytes)           int* (8 bytes)
Address:        0x7ffee14a0008          0x7ffee14a0010
Contents:       [ 42 ]                  [ 0x7ffee14a0008 ]
                                               |
                                               v
                                     Points back to value!

Structure Alignment and Hardware Padding

One of the most common surprises for systems developers is that a struct’s size in bytes is often larger than the sum of its parts. CPUs perform memory access far more efficiently when multi-byte values reside on addresses that are multiples of their size:

#include <iostream>

// Unaligned layout: Causes 7 bytes of internal padding
struct InefficientPacket {
    char flag;       // 1 byte
    // 7 bytes of padding inserted here by compiler!
    uint64_t sequence; // 8 bytes (must align to 8-byte boundary)
    char status;     // 1 byte
    // 7 bytes of tail padding inserted here!
};

// Optimized layout: Sorted by descending field alignment
struct OptimizedPacket {
    uint64_t sequence; // 8 bytes
    char flag;         // 1 byte
    char status;       // 1 byte
    // 6 bytes of tail padding (total: 16 bytes instead of 24)
};

void checkPadding() {
    std::cout << "Inefficient size: " << sizeof(InefficientPacket) << " bytes\n"; // 24 bytes
    std::cout << "Optimized size:   " << sizeof(OptimizedPacket) << " bytes\n";   // 16 bytes
}

By understanding struct alignment and keeping cache line boundaries (typically 64 bytes) in mind, you can drastically reduce memory bandwidth bottlenecks in high-throughput applications.

Conclusion

Mastering memory in C++ is not about memorizing syntax quirks. It is about understanding the boundary between hardware, operating systems, and language runtimes:

Prefer automatic stack allocation by default for predictable performance.
Use RAII and smart pointers (std::unique_ptr) whenever dynamic lifetimes are required.
Be conscious of data layout, alignment, and cache lines when designing data structures for high-performance computing.