Object Pool

Overview

Every call to new T(...) reaches the global allocator: lock acquisition, bookkeeping update, possible OS syscall, then a symmetric round-trip on delete. At low frequency this is invisible. At high frequency — game entities spawned every frame, per-request parser nodes, socket read buffers — allocation cost dominates and heap fragmentation compounds it.

An object pool sidesteps this by front-loading allocation at startup. Internally it maintains a free list: a singly-linked list threaded through the unused slots. Acquiring an object pops the list head and constructs in-place via placement new (available since C++98). Releasing runs the destructor explicitly and pushes the slot back. No allocator is called during steady-state operation.

The C++11 unrestricted union rules are the key enabler. Before C++11, a union member of class type with a non-trivial constructor was ill-formed. C++11 lifted this restriction provided constructors and destructors are user-defined — exactly what makes the Slot trick compile.

Core Implementation

template<typename T, std::size_t Capacity>
class ObjectPool {
    union Slot {                // C++11: unrestricted union
        T     object;
        Slot* next;

        Slot() noexcept : next{nullptr} {}  // must be user-defined (C++11)
        ~Slot() {}                          // trivial body — pool owns T lifetime
    };

    std::array<Slot, Capacity> slots_;
    Slot*       free_head_ = nullptr;
    std::size_t in_use_    = 0;

public:
    ObjectPool() noexcept {
        for (std::size_t i = 0; i + 1 < Capacity; ++i)
            slots_[i].next = &slots_[i + 1];
        free_head_ = &slots_[0];
    }

    // Non-copyable and non-movable: every Slot::next points into slots_
    ObjectPool(const ObjectPool&)            = delete;
    ObjectPool& operator=(const ObjectPool&) = delete;

    template<typename... Args>
    [[nodiscard]] T* acquire(Args&&... args) {  // C++17: [[nodiscard]]
        if (!free_head_) return nullptr;        // pool exhausted

        Slot* slot = free_head_;
        free_head_ = slot->next;
        ++in_use_;
        return ::new(std::addressof(slot->object)) T(std::forward<Args>(args)...);
    }

    void release(T* ptr) noexcept {
        std::destroy_at(ptr);                           // C++17; use ptr->~T() on C++14
        auto* slot = reinterpret_cast<Slot*>(ptr);
        slot->next = free_head_;
        free_head_ = slot;
        --in_use_;
    }

    [[nodiscard]] std::size_t available() const noexcept { return Capacity - in_use_; }
    [[nodiscard]] std::size_t size()      const noexcept { return in_use_; }
};

std::destroy_at (C++17) is equivalent to ptr->~T() but handles arrays cleanly and is less error-prone when T is a typedef for an array type. On C++14 and earlier, call the destructor directly.

The pool is intentionally non-copyable and non-movable: copying slots_ without patching every embedded pointer would immediately corrupt the free list. A movable pool requires a custom move constructor that rebuilds the list with adjusted addresses.

RAII Handle

Manually tracking release calls is the first step toward a bug. Wrap the raw pointer in a move-only handle:

template<typename T, std::size_t N>
class ManagedPool {
    ObjectPool<T, N> pool_;

public:
    struct Handle {
        T*           ptr_  = nullptr;
        ManagedPool* pool_ = nullptr;

        Handle() = default;
        Handle(T* p, ManagedPool* mp) noexcept : ptr_{p}, pool_{mp} {}

        ~Handle() { if (ptr_) pool_->pool_.release(ptr_); }

        Handle(const Handle&)            = delete;
        Handle& operator=(const Handle&) = delete;

        Handle(Handle&& o) noexcept
            : ptr_{std::exchange(o.ptr_, nullptr)},   // C++14: std::exchange
              pool_{std::exchange(o.pool_, nullptr)} {}

        Handle& operator=(Handle&& o) noexcept {
            if (this != &o) {
                if (ptr_) pool_->pool_.release(ptr_);
                ptr_  = std::exchange(o.ptr_,  nullptr);
                pool_ = std::exchange(o.pool_, nullptr);
            }
            return *this;
        }

        T* operator->() const noexcept { return ptr_; }
        T& operator*()  const noexcept { return *ptr_; }
        explicit operator bool() const noexcept { return ptr_ != nullptr; }
    };

    template<typename... Args>
    [[nodiscard]] Handle acquire(Args&&... args) {
        return {pool_.acquire(std::forward<Args>(args)...), this};
    }
};

ManagedPool<std::vector<int>, 32> pool;

{
    auto h = pool.acquire();
    if (!h) throw std::runtime_error("pool exhausted");
    h->push_back(42);
}  // slot returned automatically — no explicit release

Move assignment is easy to overlook. Without it, h2 = std::move(h1) fails to compile (deleted copy) unless you later add it — and adding it later is error-prone because the old value of h2 must be released first.

Thread-Safe Pool

Add a mutex and condition variable (both C++11) to serialize access across threads:

template<typename T, std::size_t N>
class ThreadSafePool {
    ObjectPool<T, N>        pool_;
    mutable std::mutex      mtx_;   // C++11
    std::condition_variable cv_;    // C++11

public:
    template<typename... Args>
    T* try_acquire(Args&&... args) {
        std::lock_guard g{mtx_};    // C++17: CTAD on lock_guard
        return pool_.acquire(std::forward<Args>(args)...);
    }

    template<typename... Args>
    T* acquire_blocking(Args&&... args) {
        std::unique_lock lock{mtx_};
        cv_.wait(lock, [this]{ return pool_.available() > 0; });
        return pool_.acquire(std::forward<Args>(args)...);
    }

    void release(T* ptr) noexcept {
        { std::lock_guard g{mtx_}; pool_.release(ptr); }
        cv_.notify_one();
    }
};

For high-contention scenarios, see the lock-free section below.

Lock-Free Free List

The mutex-protected pool serialises every acquire and release through a single lock. Under heavy concurrency this becomes a bottleneck: every thread waits for the lock even though the only shared state is the free_head_ pointer. A lock-free free list replaces the mutex with a single atomic compare-and-swap (CAS) on that pointer.

Basic CAS loop

#include <atomic>
#include <new>

template<typename T, std::size_t Capacity>
class LockFreePool {
    struct Node {
        alignas(T) unsigned char storage[sizeof(T)];
        Node* next = nullptr;
    };

    Node storage_[Capacity];
    std::atomic<Node*> head_{ nullptr };  // C++11

public:
    LockFreePool() noexcept {
        for (std::size_t i = 0; i + 1 < Capacity; ++i)
            storage_[i].next = &storage_[i + 1];
        head_.store(&storage_[0], std::memory_order_relaxed);
    }

    template<typename... Args>
    [[nodiscard]] T* acquire(Args&&... args) {
        Node* old_head = head_.load(std::memory_order_acquire);
        while (old_head) {
            Node* next = old_head->next;
            // CAS: if head_ is still old_head, swap it for next
            if (head_.compare_exchange_weak(
                    old_head, next,
                    std::memory_order_acq_rel,
                    std::memory_order_acquire))
            {
                return ::new(old_head->storage) T(std::forward<Args>(args)...);
            }
            // old_head was reloaded by compare_exchange_weak on failure
        }
        return nullptr;  // pool exhausted
    }

    void release(T* ptr) noexcept {
        ptr->~T();
        auto* node = reinterpret_cast<Node*>(
            reinterpret_cast<unsigned char*>(ptr) - offsetof(Node, storage));
        Node* old_head = head_.load(std::memory_order_relaxed);
        do {
            node->next = old_head;
        } while (!head_.compare_exchange_weak(
                     old_head, node,
                     std::memory_order_release,
                     std::memory_order_relaxed));
    }
};

compare_exchange_weak is preferred over compare_exchange_strong in loops: the weak variant may spuriously fail on architectures that use LL/SC (load-linked / store-conditional) pairs instead of a native CAS instruction (ARM, RISC-V, PowerPC). Spurious failure just re-enters the loop; the strong variant emulates success guarantees with an inner retry, which is redundant when you already loop.

The ABA problem

The CAS above is not safe as written. Consider this interleaving with two threads (T1, T2) and a free list A -> B -> C:

1. T1 reads head_ = A, computes next = B.
2. T2 acquires A, acquires B, releases A back. Free list is now A -> C.
3. T1's CAS sees head_ == A — the check passes — and sets head_ = B. But B was already acquired by T2 and is now live. The free list is silently corrupted.

This is the ABA problem: the pointer appears unchanged (still A) but the state it represents has changed. A single-word CAS cannot detect it.

Mitigation 1 — tagged pointer (version counter)

On x86-64, alignof(Node) >= 8 means the three low bits of any Node* are always zero. Pack a version counter into those bits:

struct TaggedPtr {
    uintptr_t bits;

    static TaggedPtr make(Node* p, uintptr_t tag) noexcept {
        return { reinterpret_cast<uintptr_t>(p) | (tag & 0x7u) };
    }
    Node*    ptr() const noexcept {
        return reinterpret_cast<Node*>(bits & ~uintptr_t{0x7u});
    }
    uintptr_t tag() const noexcept { return bits & 0x7u; }
};

std::atomic<TaggedPtr> head_;

On push, increment the tag; on pop, the tag embedded in old_head must still match. Any ABA cycle that completes within the same tag value remains possible, but the probability drops to 1-in-8 (or 1-in-16, etc., with more bits borrowed from alignment gaps). Eight bits of tag (requiring 256-byte alignment, feasible for large objects) makes cycling within a single slot effectively impossible.

For objects with only 8-byte alignment (the common case), the safe approach is a 128-bit double-word CAS (DWCAS) on platforms that support it:

struct alignas(16) TaggedPtr128 {
    Node*    ptr = nullptr;
    uint64_t tag = 0;
};

std::atomic<TaggedPtr128> head_;   // requires lock-free 128-bit atomics

On x86-64, std::atomic<TaggedPtr128> uses the cmpxchg16b instruction and is lock-free when is_always_lock_free is true (compiler/platform dependent — check at compile time). The 64-bit tag counter cannot realistically overflow.

Mitigation 2 — hazard pointers (C++26)

Hazard pointers are a reclamation scheme that avoids ABA by deferring reclamation: a thread publishes a hazard pointer (an atomic pointer) announcing "I am accessing this node". A reclaimer only frees a node when no hazard pointer points to it. Nodes returned to the pool are not immediately recycled — they are held in a pending list and re-queued only after all hazard pointers are cleared.

C++26 adds std::hazard_pointer to the standard library (<hazard_pointer>). Until then, folly, libcds, and similar libraries provide production-quality implementations.

When to use a lock-free pool

• Read-heavy patterns with infrequent release work well: contention on acquire is the common path; occasional release rarely conflicts.
• Lock-free pools do not compose with blocking waits. If all slots are taken, the caller must spin, back off exponentially, or fall back to a separate allocator — there is no condition variable to block on. The mutex-protected pool's acquire_blocking is simpler in that case.
• Benchmarks (e.g. the GitHub SeaTalk lock-free pool) report 16–25 million borrow-return operations per second on an 8-core machine, versus 4–6 million for a mutex-protected equivalent — a 3–4× throughput gain under high concurrency, at the cost of implementation complexity and the ABA hazard.

PMR Integration (C++17)

C++17's <memory_resource> provides std::pmr::unsynchronized_pool_resource and std::pmr::synchronized_pool_resource — standard-library pool allocators built on the same free-list idea but generalised to serve any allocation size and integrated into the polymorphic allocator chain.

How PMR pool resources work

From cppreference: each pool_resource maintains a collection of pools, one per block size. Each pool holds chunks, subdivided into equal-size blocks. When a pool is exhausted it requests a new chunk from its upstream allocator, with chunk sizes growing geometrically. Allocations that exceed the largest configured block size bypass the pools and go directly to the upstream allocator.

Key tuning knobs are exposed via std::pmr::pool_options:

struct pool_options {
    std::size_t max_blocks_per_chunk = 0;  // 0 = implementation default
    std::size_t largest_required_pool_block = 0;  // 0 = implementation default
};

Chaining with monotonic_buffer_resource

A common pattern stacks a pool resource on top of a monotonic_buffer_resource so that the upstream never calls ::operator new:

#include <memory_resource>

// Stack-allocated backing store — zero heap traffic
alignas(std::max_align_t) std::byte buf[64 * 1024];
std::pmr::monotonic_buffer_resource mono{ buf, sizeof(buf) };

std::pmr::pool_options opts;
opts.largest_required_pool_block = 256;  // objects up to 256 bytes use pools
std::pmr::unsynchronized_pool_resource pool{ opts, &mono };

// PMR-aware containers use the pool
std::pmr::vector<std::pmr::string> lines{ &pool };
lines.emplace_back("hello");  // allocates from pool, not heap

When pool is destroyed or pool.release() is called, all memory is bulk-freed back to mono. When mono goes out of scope, the stack buffer is simply abandoned — no per-object destructors required for the memory itself.

unsynchronized vs synchronized

synchronized_pool_resource provides the same interface as its unsynchronized sibling but acquires an internal lock on each allocation/deallocation — equivalent to wrapping unsynchronized_pool_resource in a mutex. For fine-grained control over locking, prefer per-thread unsynchronized_pool_resource instances and eliminate sharing entirely.

PMR pools vs hand-rolled ObjectPool

Use pmr::pool_resource when you need standard containers (pmr::vector, pmr::map, pmr::string) to live in a pool, or when you cannot template on the allocator. Use the custom ObjectPool when you need typed acquire/release with RAII, a hard capacity bound, or when virtual dispatch overhead matters.

Cache Behaviour and Pool Warming

Object pools deliver two distinct performance wins. The first — eliminating allocator overhead — is obvious. The second — cache locality — is subtler and often larger in practice.

Why malloc returns cold memory

The OS and allocator hand out pages on demand. A freshly malloc'd pointer may map to a physical page that has never been touched. The first read triggers a minor page fault (or TLB miss at minimum), and the cache line holding that address is cold — it must be fetched from DRAM. Each DRAM fetch takes roughly 60–100 ns; an L1 hit takes ~1 ns and an L2 hit ~5 ns. An object pool pre-touches its entire storage array at construction time, converting compulsory misses into L1 or L2 hits for all subsequent accesses.

Beyond cold-start misses, malloc suffers capacity and conflict misses. It may coalesce free blocks from many different allocation sites, scattering the returned addresses across disparate cache sets. An object pool allocates contiguously from std::array<Slot, N> — successive acquires return addresses that differ by exactly sizeof(Slot), maximising spatial locality.

Quantifying the benefit

The three Cs of cache misses (from Hennessy & Patterson) apply directly:

• Compulsory misses: one miss per cache line, per pool, at pool construction — paid once at startup, not in the hot path.
• Capacity misses: a working set of N pool objects fits in N * sizeof(T) bytes of contiguous memory. The same objects allocated via new may be scattered across the heap, increasing the effective working-set footprint by 2–4× due to allocator metadata and fragmentation.
• Conflict misses: contiguous layout reduces the chance that two hot objects map to the same cache set; scattered heap objects have no such guarantee.

Practical measurements from game-engine and server workloads (cited in Stack Overflow Q#7319774 and various GDC talks) consistently show pool allocation delivering a 2–5× reduction in L1 data cache miss rates for object-heavy loops, with the benefit growing with object count and shrinking with object size (large objects always miss regardless of layout).

Pool warming strategy

template<typename T, std::size_t N>
class ObjectPool {
    // ... (as above)

    // Touch every cache line of the backing store at construction time
    // so physical pages are mapped and the first acquire is L1/L2 warm.
    void warm_cache() noexcept {
        volatile unsigned char* p =
            reinterpret_cast<volatile unsigned char*>(slots_.data());
        const std::size_t total = sizeof(slots_);
        // One write per cache line (typically 64 bytes)
        for (std::size_t i = 0; i < total; i += 64)
            p[i] = 0;
    }

public:
    ObjectPool() noexcept {
        for (std::size_t i = 0; i + 1 < Capacity; ++i)
            slots_[i].next = &slots_[i + 1];
        free_head_ = &slots_[0];
        warm_cache();
    }
};

The volatile write prevents the compiler from eliding the loop as dead store. An alternative is std::memset(slots_.data(), 0, sizeof(slots_)), which achieves the same page-fault elimination and is guaranteed not to be eliminated by the compiler.

For pools larger than L2 (typically >256 KB), warming at construction does not help for the common case — the cache will have evicted cold lines long before they are needed. In that regime, a slab approach (carve a small active pool out of a large backing store and refill from the backing store when the active pool drains) keeps the active working set L2-resident.

std::construct_at and std::destroy_at (C++17/C++20)

Before C++17, the idiomatic way to construct an object in pre-allocated storage was a raw placement-new expression:

T* ptr = ::new(static_cast<void*>(buf)) T(args...);

C++17 introduced std::destroy_at(ptr) as a named wrapper for ptr->~T(). C++20 added std::construct_at(ptr, args...) as a named wrapper for placement new.

#include <memory>  // std::construct_at, std::destroy_at (C++20/C++17)

alignas(T) unsigned char buf[sizeof(T)];
T* ptr = std::construct_at(reinterpret_cast<T*>(buf), arg1, arg2);
// ... use *ptr ...
std::destroy_at(ptr);

Why prefer these over raw placement new

Constant evaluation: ::new is not permitted in a constexpr context, but std::construct_at is — it participates in constant evaluation when the underlying type is constructible at compile time. This makes constexpr object pools possible in C++20.

Array types: std::destroy_at(ptr) correctly handles array types by calling the destructor on each element. ptr->~T() applied to a T[] array pointer is ill-formed.

Readability and tooling: Named functions are greppable, appear in stack traces, and interact better with sanitisers than raw placement-new syntax.

In the pool context

Replace the raw placement-new in acquire and the raw destructor call in release:

// C++20 version of acquire's construction line:
return std::construct_at(std::addressof(slot->object),
                         std::forward<Args>(args)...);

// C++17 version of release's destruction line:
std::destroy_at(ptr);   // already used above; unchanged

std::construct_at is constexpr and requires #include <memory> (already pulled in by most codebases). On compilers that lack full C++20 support, provide a polyfill:

#if __cplusplus < 202002L
namespace std {
    template<typename T, typename... Args>
    T* construct_at(T* p, Args&&... args) {
        return ::new(static_cast<void*>(p)) T(std::forward<Args>(args)...);
    }
}
#endif

Best Practices

Pre-size from measurements, not guesses. Profile the peak simultaneous live count under representative load, then add 20–30% headroom. Guessing leads to either wasted memory or silent nullptr returns under production load.

Ensure the pool outlives all handles. Handle::~Handle() calls back into the pool. If the pool is destroyed first, the destructor dereferences a dangling pointer. Pools should be static, global, or owned by an object with strictly longer lifetime than any borrower.

Mark acquire with [[nodiscard]] (C++17). Discarding the return value orphans a slot permanently. The attribute makes this a compiler warning.

Test the exhaustion path. acquire returning nullptr is a legitimate steady-state condition. Callers that unconditionally dereference the result will crash under load. Write tests that fill the pool to capacity.

Common Pitfalls

Use-after-release. Calling release(ptr) runs the destructor and makes ptr a dead pointer. Any subsequent dereference is undefined behaviour. The RAII handle prevents this entirely; raw pointer APIs cannot. In debug builds, overwrite the released slot with a sentinel byte pattern to catch stale reads early.

Double-release. Releasing the same pointer twice corrupts the free list, typically manifesting as a crash or silent data corruption far from the source. Instrument release in debug builds with an std::unordered_set<T*> of live pointers and assert before each release.

Cross-pool release. Two ObjectPool instances hold separate slots_ arrays. Releasing a pointer obtained from pool A into pool B inserts a foreign address into B's free list and corrupts it on the next acquire. Ownership must be unambiguous; the RAII handle carries the pool pointer to prevent this.

Destroying the pool with live handles. All pending Handle destructors will call release on a destroyed pool's storage. Make destruction order explicit — either through RAII scope ordering or by asserting size() == 0 in the pool's destructor.

See Also

• std::pmr::pool_resource (C++17) — standard pool allocator for PMR-aware containers
• std::construct_at (C++20) — placement-new wrapper that participates in constant evaluation
• std::destroy_at (C++17) — explicit destruction without deallocation
• RAII — the ownership model that makes pool handles leak-safe
• std::allocator_traits (C++11) — interface contract for writing custom allocators