Master Advanced C++ Multithreading: Atomics, Memory Ordering, and Lock-Free Design

By the end of this page, you will understand the C++ memory model and why it matters, use std::atomic with explicit memory ordering to build correct concurrent code, recognize when acquire/release semantics are sufficient versus sequentially consistent, implement a lock-free data structure, and leverage C++20's std::latch, std::barrier, and std::jthread for cleaner thread coordination.

What and Why

Most C++ engineers reach for a std::mutex and call it done. Mutexes are correct, composable, and easy to reason about — but they serialize execution at the hardware level through expensive cache-line invalidation and kernel transitions. When you need maximum throughput in a hot path, you need to understand what actually happens at the memory level.

The C++ memory model defines the rules by which operations in one thread become visible to other threads. Without explicit synchronization, the compiler and CPU are both free to reorder instructions in ways that are locally valid but globally surprising. This is not a bug in your toolchain — it is a deliberate contract that enables performance on modern out-of-order and multi-core processors.

std::atomic<T> is the primary tool for crossing that boundary safely. It guarantees that reads and writes to the atomic variable are indivisible, and — critically — it lets you choose how much ordering you need, paying only for what you use.

C++20 is required for std::jthread, std::latch, and std::barrier. Atomic memory ordering has been available since C++11.

Step by Step

The problem with naive shared state

#include <thread>
#include <iostream>

int counter = 0; // plain int, not atomic

int main() {
    std::thread t1([] { for (int i = 0; i < 100'000; ++i) ++counter; });
    std::thread t2([] { for (int i = 0; i < 100'000; ++i) ++counter; });
    t1.join();
    t2.join();
    std::cout << counter << '\n'; // undefined behavior; result varies
}

The increment ++counter compiles to a read-modify-write sequence. Two threads can interleave those steps, losing updates silently.

Fixing it with std::atomic — default ordering

#include <atomic>
#include <thread>
#include <iostream>

std::atomic<int> counter{0};

int main() {
    std::thread t1([] { for (int i = 0; i < 100'000; ++i) counter.fetch_add(1); });
    std::thread t2([] { for (int i = 0; i < 100'000; ++i) counter.fetch_add(1); });
    t1.join();
    t2.join();
    std::cout << counter.load() << '\n'; // always 200000
}

The default memory order is std::memory_order_seq_cst — the strongest and most expensive guarantee. Every sequentially consistent operation establishes a single total order visible to all threads.

Relaxing the ordering: acquire/release

For a producer–consumer flag, you do not need a global total order. You only need the producer's writes to be visible to the consumer once the flag is set. This is the acquire/release contract:

#include <atomic>
#include <thread>
#include <string>
#include <cassert>

std::atomic<bool> ready{false};
std::string data; // non-atomic payload

void producer() {
    data = "hello from producer"; // (1) write payload
    // release: everything written before this store
    // is visible to any thread that later acquires this flag
    ready.store(true, std::memory_order_release); // (2)
}

void consumer() {
    // acquire: don't hoist any reads past this load
    while (!ready.load(std::memory_order_acquire)) {} // (3) spin
    assert(data == "hello from producer"); // (4) safe: (1) happened-before (4)
}

int main() {
    std::thread t1(producer);
    std::thread t2(consumer);
    t1.join();
    t2.join();
}

The key insight: release and acquire on the same atomic variable create a happens-before edge. Everything written before the release store is guaranteed visible after the acquire load succeeds. No global ordering is established — just a directed edge between these two threads, which is cheaper.

std::memory_order_relaxed provides no ordering at all — only atomicity. Use it for statistics counters where a slightly stale read is acceptable.

Lock-free stack

Armed with acquire/release, here is a minimal lock-free Treiber stack:

#include <atomic>
#include <memory>
#include <optional>

template<typename T>
class LockFreeStack {
    struct Node {
        T value;
        Node* next;
    };
    std::atomic<Node*> head_{nullptr};

public:
    void push(T val) {
        Node* node = new Node{std::move(val), nullptr};
        node->next = head_.load(std::memory_order_relaxed);
        // compare_exchange_weak retries on spurious failure (preferred in loops)
        while (!head_.compare_exchange_weak(node->next, node,
                                            std::memory_order_release,
                                            std::memory_order_relaxed)) {}
    }

    std::optional<T> pop() {
        Node* old = head_.load(std::memory_order_acquire);
        while (old && !head_.compare_exchange_weak(old, old->next,
                                                   std::memory_order_acquire,
                                                   std::memory_order_acquire)) {}
        if (!old) return std::nullopt;
        T val = std::move(old->value);
        delete old; // safe only if no other thread holds 'old' — see ABA note below
        return val;
    }
};

Note: This stack has the ABA problem and unsafe deletion in a real multi-reader scenario. Production lock-free code requires hazard pointers or epoch-based reclamation. This example demonstrates the pattern; always add memory reclamation before shipping.

C++20 synchronization primitives

std::latch is a single-use counter for fan-in: threads decrement it, one waiter unblocks when it reaches zero.

#include <latch>
#include <thread>
#include <vector>
#include <iostream>

int main() {
    constexpr int N = 4;
    std::latch all_ready{N};

    std::vector<std::jthread> workers;
    for (int i = 0; i < N; ++i) {
        workers.emplace_back([&, i] {
            // simulate setup work
            std::this_thread::sleep_for(std::chrono::milliseconds(i * 10));
            std::cout << "worker " << i << " ready\n";
            all_ready.count_down(); // decrement; last one unblocks the waiter
        });
    }

    all_ready.wait(); // blocks until count reaches zero
    std::cout << "all workers ready, starting main task\n";
} // jthread destructor joins automatically

std::barrier is reusable — after all threads arrive, it resets and can be used again. It also accepts a completion callback run by exactly one thread at each sync point, which is ideal for phase-based pipelines.

std::barrier with Completion Callback (C++20)

The completion function passed to std::barrier runs exactly once per phase — on whichever thread happens to bring the count to zero — before any waiting thread is released. This makes it a natural hook for inter-phase work such as swapping buffers or logging progress, without any additional synchronization.

#include <barrier>
#include <iostream>
#include <vector>
#include <array>
#include <thread>
#include <syncstream>

int main() {
    constexpr int N_THREADS = 4;
    constexpr int N_PHASES  = 3;

    // Two ping-pong buffers swapped between phases
    std::array<std::vector<int>, 2> buffers{
        std::vector<int>(N_THREADS, 0),
        std::vector<int>(N_THREADS, 0)
    };
    int active = 0; // which buffer workers write into

    int phase_number = 0;

    // Completion callback: runs once per phase, on one thread, before release
    auto on_phase_done = [&]() noexcept {
        // swap buffers so the next phase writes into the idle one
        active ^= 1;
        std::cout << "=== phase " << ++phase_number << " complete ===\n";
    };

    std::barrier sync{N_THREADS, on_phase_done};

    std::vector<std::jthread> workers;
    workers.reserve(N_THREADS);

    for (int id = 0; id < N_THREADS; ++id) {
        workers.emplace_back([&, id] {
            for (int phase = 0; phase < N_PHASES; ++phase) {
                // Each worker writes into the active buffer
                buffers[active][id] = (phase + 1) * (id + 1);
                std::osyncstream(std::cout)
                    << "  thread " << id << " phase " << phase
                    << " wrote " << buffers[active][id] << '\n';

                // arrive_and_wait: signal arrival and block until all N arrive
                // The last arrival triggers on_phase_done before releasing everyone
                sync.arrive_and_wait();
            }
        });
    }
} // jthread destructors join

Key rules for the completion function: it must be noexcept, it must not call any member of the same barrier, and it runs strongly happens-before all threads are released from wait(), so any writes it makes are visible to all workers once they continue.

Atomic Wait and Notify (C++20)

C++20 adds wait(), notify_one(), and notify_all() directly to std::atomic<T>. They let one thread block until the atomic's value changes, without a mutex or condition variable. The implementation may use OS futexes or WaitOnAddress, making it far cheaper than a spin loop and lighter-weight than a condition_variable when no mutex is needed.

#include <atomic>
#include <thread>
#include <string>
#include <iostream>

std::atomic<bool> data_ready{false};
std::string payload;

int main() {
    std::jthread producer([&] {
        payload = "result computed";
        data_ready.store(true, std::memory_order_release);
        // Wake exactly one thread blocked in data_ready.wait(false)
        data_ready.notify_one();
    });

    // Block until data_ready != false (i.e., until it becomes true).
    // Internally checks the current value; if it already equals the argument,
    // it parks the thread and re-checks on each notify.
    data_ready.wait(false, std::memory_order_acquire);
    std::cout << payload << '\n'; // safe: release/acquire pair established
}

Contrast this with the acquire/release spin shown earlier: wait() parks the calling thread in the OS instead of burning CPU. notify_all() wakes every waiter — use it when multiple consumers are waiting on the same flag.

Important caveats:

• wait(old) compares bitwise. It wakes when the stored value no longer equals old, which means if the value changes from old to X and back to old (the ABA sequence), some waiters may never wake.
• Valid memory orders for wait() are relaxed, consume, acquire, and seq_cst. Release-family orders are not permitted on a load.
• The notification calls have no memory-order parameter; they always act as seq_cst releases.

std::atomic_ref (C++20)

std::atomic_ref<T> projects atomic operations onto an existing, non-atomic variable without changing its type. While the atomic_ref handle is alive, every access to the underlying object must go through some atomic_ref instance — mixing plain and atomic access is undefined behavior just as it is with std::atomic.

The canonical use case is applying atomic operations to a member of a third-party struct whose definition you cannot modify:

#include <atomic>
#include <thread>
#include <vector>
#include <iostream>

struct Sensor {
    // Cannot change this layout — it's from an external library
    int reading{0};
    float temperature{0.0f};
};

int main() {
    Sensor s;

    // Atomically increment s.reading from multiple threads
    // without changing the type of Sensor::reading
    std::vector<std::jthread> workers;
    for (int i = 0; i < 8; ++i) {
        workers.emplace_back([&s] {
            std::atomic_ref<int> aref{s.reading};
            aref.fetch_add(1, std::memory_order_relaxed);
        });
    }
    workers.clear(); // joins all threads

    // After all atomic_ref handles are gone, plain access is safe again
    std::cout << "total readings: " << s.reading << '\n'; // always 8
}

std::atomic_ref<T> requires T to be trivially copyable, just like std::atomic<T>. It supports the same full interface: load, store, fetch_add, compare_exchange_weak, wait/notify_one, etc. One additional constraint: the object must be aligned to at least std::atomic_ref<T>::required_alignment — on most platforms this is the natural alignment of T, so plain int and pointer members are fine.

atomic_ref is also useful for arrays: you can slice a plain std::vector<int> and let different threads own disjoint elements atomically, avoiding a std::vector<std::atomic<int>> which would break resize and serialized initialization.

Common Patterns

Double-checked locking with atomics (C++11+)

#include <atomic>
#include <mutex>
#include <memory>

class Config {
    static std::atomic<Config*> instance_;
    static std::mutex init_mutex_;
public:
    static Config* get() {
        Config* p = instance_.load(std::memory_order_acquire);
        if (!p) {
            std::lock_guard lock(init_mutex_);
            p = instance_.load(std::memory_order_relaxed);
            if (!p) {
                p = new Config();
                instance_.store(p, std::memory_order_release);
            }
        }
        return p;
    }
};
std::atomic<Config*> Config::instance_{nullptr};
std::mutex Config::init_mutex_;

Signaling between threads with std::binary_semaphore (C++20)

A semaphore is not mutex-tied: the thread that calls release() does not need to be the same thread that called acquire(). This makes it ideal for one-directional signaling — initialize to 0 so the consumer blocks immediately, then release from the producer when data is ready.

#include <semaphore>
#include <thread>
#include <iostream>
#include <string>

std::binary_semaphore smph{0};  // starts at 0 — consumer will block
std::string payload;

int main() {
    std::jthread producer([&] {
        payload = "data is ready";
        smph.release();    // unblocks the consumer
    });

    smph.acquire();        // blocks until producer calls release()
    std::cout << payload << '\n';
}

std::binary_semaphore is an alias for std::counting_semaphore<1> — its internal count is either 0 or 1. std::counting_semaphore<N> allows up to N simultaneous acquires, useful for throttling a pool of workers to a fixed concurrency limit:

// Allow at most 3 concurrent database connections
std::counting_semaphore<3> db_slots{3};

void query(int id) {
    db_slots.acquire();         // blocks when all 3 slots are taken
    do_db_work(id);
    db_slots.release();         // returns a slot
}

Unlike a mutex, semaphore ownership is not tied to a thread — any thread can call release(). Unlike a condition variable, there is no spurious-wakeup problem and no mutex pairing required.

Cooperative cancellation with std::stop_token (C++20)

#include <thread>
#include <stop_token>
#include <iostream>

int main() {
    std::jthread worker([](std::stop_token token) {
        while (!token.stop_requested()) {
            // do incremental work
            std::this_thread::sleep_for(std::chrono::milliseconds(50));
        }
        std::cout << "worker cancelled cleanly\n";
    });

    std::this_thread::sleep_for(std::chrono::milliseconds(200));
    worker.request_stop(); // signals the stop_token; jthread joins on destruction
}

What Can Go Wrong

Using memory_order_relaxed for a flag. If the consumer spins on a relaxed load of your ready flag, the compiler is free to hoist the load out of the loop entirely — it sees no observable side effects requiring a re-read. Always use at least memory_order_acquire on the consumer side of a flag.

Mixing std::atomic access with plain access. Accessing the same variable through both an atomic and a non-atomic reference is undefined behavior. Every access to a shared variable must go through the same atomic object.

Assuming compare_exchange_strong is always better than weak. In a retry loop, compare_exchange_weak is preferred because on architectures with LL/SC instructions (ARM, RISC-V), the strong variant emits an internal retry loop that doubles the loop overhead. Use strong only when a spurious failure would be semantically incorrect (i.e., outside a loop).

Over-synchronizing with seq_cst. Sequentially consistent operations on x86 compile to MFENCE or LOCK XCHG, which stall the CPU pipeline. Profile before reaching for the default; acquire/release is almost always sufficient for producer–consumer patterns.

Quick Reference

• Happens-before is established by a release store followed by an acquire load on the same atomic.
• std::atomic<T> requires T to be trivially copyable.
• On x86, acquire/release are free (the hardware is TSO); they cost only a compiler fence. On ARM they emit real barriers.

What's Next

• Coroutines and async I/O — suspend-and-resume as an alternative to thread-per-task models
• Memory model deep dive — how value semantics and move interact with concurrent ownership
• Lambda captures in concurrent code — lifetime hazards when lambdas outlive their enclosing scope
• Lock-free memory reclamation — hazard pointers and epoch-based reclamation to make the lock-free stack production-ready