On this page

Memory Allocators

Every time you call malloc() in C, new in C++, or allocate an object in a garbage-collected language, a memory allocator decides where that memory comes from. The allocator manages the heap: tracking free regions, splitting and coalescing blocks, and deciding when to ask the kernel for more memory. Poor allocator design causes fragmentation, lock contention, and mysterious performance cliffs. Good allocator design — like jemalloc in Firefox and Meta's servers, or tcmalloc in Chrome and Google — is a meaningful competitive advantage.

The allocator sits in a gap that most engineers never look at: below your language, above the kernel. It is responsible for some of the most confusing production behaviour you will see — a service whose RSS grows for days and never shrinks even though its "live data" is flat, a multi-threaded program that gets slower when you add cores, a container OOM-killed with a heap well under its limit, or memory that drops by 40% after a one-line LD_PRELOAD change. This page explains how allocators work from first principles, how the major production allocators differ, and how to diagnose allocator problems with real tools.

Prerequisites

Read Virtual Memory first — especially demand paging, mmap, huge pages, and RSS vs PSS. Allocators are built entirely on those mechanisms.


Where the Allocator Sits

There are two allocators in every Linux process's life: the user-space allocator (malloc, part of libc or a replacement library) and the kernel's allocators (the buddy page allocator and the slab allocator). They solve the same problem at different granularities.

flowchart TD
    APP["Application: malloc(48), new Foo, make([]byte, n)"] --> UA["User-space allocator (ptmalloc, jemalloc, tcmalloc, mimalloc, Go runtime)"]
    UA -->|"brk / mmap / madvise"| SYS["System calls"]
    SYS --> VMA["Kernel VM: VMAs and page tables"]
    VMA -->|"page fault on first touch"| BUDDY["Buddy page allocator (4KB..4MB blocks)"]
    KOBJ["Kernel objects: inodes, sockets, task_struct"] --> SLUB["SLUB slab allocator"]
    SLUB --> BUDDY
    BUDDY --> RAM["Physical frames"]

Two consequences matter constantly in practice:

  1. The user-space allocator asks the kernel for big chunks and carves them up. Syscalls cost hundreds of nanoseconds; a good malloc costs 10–20 ns. So almost all malloc/free calls never reach the kernel.
  2. Freeing memory in your program is not the same as returning it to the OS. free() gives a block back to the allocator. Whether the allocator gives the underlying pages back to the kernel (munmap, madvise(MADV_DONTNEED), brk shrink) is a separate policy decision — and the source of most "my RSS never goes down" confusion.

The Heap and brk()/mmap()

The heap is a region of virtual memory that grows upward as the program allocates more. The kernel exposes two ways to get more memory:

  • brk(addr) — moves the end of the heap segment to addr. Cheap, contiguous. Only works for growing/shrinking the heap monotonically.
  • mmap(MAP_ANONYMOUS) — maps a new anonymous page anywhere in the address space. More flexible; used for large allocations.

malloc manages a pool of memory obtained from these syscalls. For small allocations, it carves up a larger slab. For large allocations (typically > 128KB), it directly calls mmap and returns the mapping.

You can watch this happen:

// heap_demo.c — gcc -O0 heap_demo.c -o heap_demo && strace -e trace=brk,mmap,munmap ./heap_demo
#include <stdlib.h>
#include <stdio.h>

int main(void) {
    void *small = malloc(100);          // served from the brk heap
    void *big   = malloc(1 << 20);      // 1 MiB > threshold: its own mmap
    printf("small=%p big=%p\n", small, big);
    free(big);                          // munmap immediately
    free(small);                        // stays in the heap for reuse
    return 0;
}
brk(NULL)                               = 0x5583f1a6a000
brk(0x5583f1a8b000)                     = 0x5583f1a8b000     ← heap grows by 132KB for a 100-byte malloc
mmap(NULL, 1052672, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9a2c5ff000
small=0x5583f1a6a2a0 big=0x7f9a2c5ff010
munmap(0x7f9a2c5ff000, 1052672)         = 0

Notice the first brk grows the heap by 132KB (the 128KB M_TOP_PAD default plus rounding) even though we asked for 100 bytes — amortising future syscalls. The 1 MiB request gets its own mapping, 4KB bigger than requested to hold the chunk header and round to a page.

glibc's thresholds

TunableDefault (64-bit)Meaning
M_MMAP_THRESHOLD128 KiB, dynamic up to 32 MiBRequests at least this big go straight to mmap
M_TRIM_THRESHOLD128 KiB (doubles with the dynamic mmap threshold)When free space at the top of the brk heap exceeds this, shrink the heap with brk
M_TOP_PAD128 KiBExtra slack added when growing the heap
M_ARENA_MAX8 × CPU coresMaximum number of arenas
M_MMAP_MAX65536Maximum simultaneous mmap'd chunks

The dynamic threshold is a subtle trap: when you free() an mmap'd chunk, glibc raises the threshold to that chunk's size, on the theory that you'll allocate similar sizes again and they're better served from the heap. A program that allocates and frees a few 10MB buffers early will have all later 10MB buffers served from arenas — where they can fragment and may never be returned. Setting the threshold explicitly (mallopt(M_MMAP_THRESHOLD, ...) or GLIBC_TUNABLES=glibc.malloc.mmap_threshold=131072) disables the dynamic adjustment.

brk heapmmap per allocation
Syscall per allocNo (amortised)Yes — mmap + munmap each, plus page faults
Returned to OS on freeOnly from the top, past the trim thresholdImmediately
FragmentationPossibleNone (but 4KB-granular internal waste)
Page-fault costOnce per page, reused afterwardsEvery time: fresh zeroed pages for every allocation

How a Simple malloc Works

A naive malloc maintains a free list — a linked list of free memory blocks, each containing a header with size and a next pointer.

Heap memory:
[hdr|  16 bytes free  |hdr|  32 bytes used  |hdr|  64 bytes free  ]
  ↑
free list: head → block1(16) → block3(64) → NULL

malloc(n) steps:

  1. Scan free list for a block ≥ n bytes (first-fit, best-fit, or worst-fit strategies)
  2. If found: split the block (return n bytes, keep remainder in free list)
  3. If not found: call brk()/mmap() to get more memory from the kernel

free(ptr) steps:

  1. Mark the block as free (restore it to the free list)
  2. Attempt to coalesce adjacent free blocks into one larger block (prevents fragmentation)

How does free(ptr) know the size of the block, when you never told it? The header sits just before the pointer malloc returned. And how does it coalesce with the previous block in O(1)? Boundary tags: the size is stored at both ends of a free block (or, in glibc, the previous chunk's size is stored at the start of the next chunk while the previous chunk is free), so you can step backwards.

Here is glibc's real chunk layout on 64-bit:

            +--------------------------------------------+
chunk  ->   | prev_size (only meaningful if prev is free) |  8 bytes
            +--------------------------------------------+
            | size of this chunk                | A | M | P |  8 bytes; low 3 bits are flags
            +--------------------------------------------+
mem    ->   | user data ...                               |  ← pointer returned by malloc
            | (when free: fd, bk pointers for the bin list)|
            +--------------------------------------------+
next chunk  | prev_size = this chunk's size (if free)     |
  • P (PREV_INUSE): the previous chunk is allocated (so don't coalesce backwards)
  • M (IS_MMAPPED): this chunk came from its own mmap
  • A (NON_MAIN_ARENA): this chunk belongs to a secondary arena

Sizes are multiples of 16, so the low bits are free for flags. The minimum chunk is 32 bytes. That gives the classic per-allocation overhead numbers: malloc(1) through malloc(24) use a 32-byte chunk; malloc(25) uses 48. If your program allocates 100 million 8-byte objects individually, you are using 3.2GB to store 800MB.

#include <malloc.h>
#include <stdio.h>
int main(void) {
    for (size_t n = 1; n <= 64; n += 7)
        printf("malloc(%2zu) -> usable %zu\n", n, malloc_usable_size(malloc(n)));
}
// malloc( 1) -> usable 24   malloc( 8) -> usable 24   malloc(22) -> usable 24
// malloc(29) -> usable 40   malloc(36) -> usable 40   malloc(50) -> usable 56 ...

Note the usable size of a 32-byte chunk is 24, not 16: an allocated chunk can use the next chunk's prev_size field, since that field is only meaningful when this chunk is free.


Fragmentation

Fragmentation is the primary enemy of a memory allocator.

External Fragmentation

Free memory exists but is split into small non-contiguous pieces — too small to satisfy a large allocation.

Heap: [free 16][used 32][free 16][used 32][free 16]
                                              ↑
Available free: 48 bytes total, but max contiguous = 16 bytes
malloc(32) fails — even though 48 bytes are free

Solutions: Coalescing adjacent free blocks; compaction (impossible if pointers exist); segregated free lists (keep separate pools per size class).

(In a real malloc the request doesn't actually fail — the allocator just grows the heap — so external fragmentation shows up as RSS that is much larger than live data, not as NULL returns.)

Internal Fragmentation

A block is allocated larger than requested (due to alignment or size rounding). The waste is inside allocated blocks.

malloc(9) on a 16-byte aligned allocator → 16-byte block allocated
7 bytes wasted inside the allocated block

Fragmentation in real servers: the "pinned top" problem

The textbook pictures above are about single allocations. In long-running servers the far more expensive effect is pages that can't be returned to the OS because one live object sits on them.

brk heap after a traffic spike (each box = one 4KB page):
[A][A][A][A][A][A][A][A][A][A][A][A]   ← all allocated during the spike

after the spike, 95% of objects freed, but a few long-lived ones remain:
[A][ ][ ][ ][A][ ][ ][ ][ ][ ][ ][S]
 ↑           ↑                    ↑
 cache entry  session              one long-lived object at the TOP of the heap
  • glibc's main arena can only shrink with brk from the top. One live object near the top (S) pins everything below it.
  • Free pages in the middle could be returned with madvise(MADV_DONTNEED), but glibc only does this in malloc_trim(), which it rarely calls automatically.
  • Result: RSS stays at the spike's high-water mark forever, though live data is 5% of it.

This is why the same Ruby, Python or Java-native workload can use 2× the memory on glibc that it uses on jemalloc: jemalloc and tcmalloc track memory in page-sized runs and purge free pages back to the OS on a timer regardless of position.

Measure fragmentation as a ratio:

ToolMetricHealthy
Redis INFO memorymem_fragmentation_ratio = RSS / used_memory1.0–1.5; > 1.5 investigate; < 1.0 means swapping
jemalloc stats.resident / stats.allocatedresident vs requested< 1.3 typical
glibc malloc_info()<system type="current"> vs sum of in-use—
Go runtime.MemStatsHeapInuse − HeapAlloc (span fragmentation), HeapIdle − HeapReleased (retained)small

Redis active defragmentation

Redis (with jemalloc) can fix fragmentation online: activedefrag yes scans keys, asks jemalloc whether the allocation sits in a poorly utilised run (je_get_defrag_hint), and if so reallocates and copies the value to a denser region, freeing the old run. That only works because Redis owns every pointer to its values — a general-purpose program can't move objects, which is exactly why C/C++ allocators can't compact.


The Buddy System

Used by the Linux kernel for page-level allocation. Memory is divided into blocks of power-of-two sizes. Each block has a "buddy" — the block of the same size immediately adjacent.

Allocation of size N (rounded up to next power of 2):

  1. Find the smallest free block ≥ N
  2. If it's larger than needed, split it into two buddies; place one in the free list
  3. Repeat until you have a block of the right size

Deallocation:

  1. Free the block
  2. Check if its buddy is also free
  3. If so, merge them into a block of double size
  4. Repeat until buddy is not free or max size reached
Splitting a 64-byte block to satisfy a 16-byte request:
64 → [32 | 32] → [16 | 16 | 32] → allocate left 16
Free list: 16(buddy), 32

Advantage: O(log N) allocation and deallocation, guaranteed coalescing. Disadvantage: Internal fragmentation — any allocation is rounded up to a power of 2. Requesting 33 bytes wastes 31 bytes (rounded to 64).

The elegance is in finding the buddy: for a block of size 2ᵏ at address a (aligned to 2ᵏ), its buddy is at a XOR 2ᵏ. No search needed.

The buddy allocator in Linux

In the kernel the unit is a page, and sizes are orders: order 0 = 1 page (4KB), order 1 = 2 pages, … order 10 = 1024 pages (4MB), the default maximum (MAX_PAGE_ORDER). Free lists are kept per NUMA node, per zone (DMA, DMA32, Normal), and per migrate type (movable, unmovable, reclaimable) so that unmovable kernel allocations don't get scattered across memory that could otherwise be compacted.

$ cat /proc/buddyinfo
Node 0, zone      DMA      0      0      0      0      0      0      0      0      1      1      2
Node 0, zone    DMA32   1543    882    517    310    178     97     45     21      9      4    310
Node 0, zone   Normal  48211  20117   6012   1180    212     41      3      0      0      0      0
#                     order0  order1 ...                                              order10

The Normal line above is a fragmented machine: tens of thousands of single free pages, but zero free blocks of order 7 or higher. A request for a 2MB huge page (order 9) must first run compaction — migrating movable pages to assemble a contiguous block — which is the root cause of the THP latency spikes described in Virtual Memory.

cat /sys/kernel/debug/extfrag/extfrag_index     # near 1.0 = failures due to fragmentation
echo 1 > /proc/sys/vm/compact_memory            # force a full compaction (careful in prod)
grep -E 'compact_(stall|fail|success)' /proc/vmstat

Per-CPU page lists (pcp) sit in front of the buddy allocator so that most order-0 allocations avoid the zone lock entirely.


Slab Allocator

The Linux kernel's primary allocator for kernel objects. Addresses the problem of frequently allocating and freeing objects of the same size (task structs, inodes, dentries, socket buffers).

Key insight: Instead of generic memory, maintain pools of pre-allocated, pre-initialized objects of each common type.

Slab cache for struct inode (size = 592 bytes):
  Slab 1: [inode][inode][inode][inode] ... 8 inodes, 2 free
  Slab 2: [inode][inode][inode][inode] ... 8 inodes, 8 free
  Slab 3: empty (returned to page allocator)

alloc_inode(): grab from Slab 1's free list → O(1)
free_inode():  return to Slab 1's free list → O(1)

Advantages:

  • No splitting/coalescing (fixed-size objects)
  • Objects can retain partial initialization between uses (constructor called once, destructor skipped)
  • Cache-friendly — objects of the same type are collocated

/proc/slabinfo shows all slab caches, their object sizes, and usage.

SLAB, SLUB and SLOB

The design above is Jeff Bonwick's slab allocator (SunOS 5.4, 1994), and Linux has had three implementations of the idea:

ImplementationStatusNotes
SLABRemoved in 6.8 (2024)Original Linux port; per-CPU arrays plus complex queues per node
SLOBRemoved in 6.4 (2023)Tiny-memory embedded variant
SLUBDefault since 2.6.23, now the only oneSimpler: per-CPU "active slab", free list threaded through the objects themselves, no per-slab metadata header, merging of caches with compatible sizes

SLUB's fast path is a per-CPU pointer to a slab and a lockless cmpxchg on its free list — tens of nanoseconds. Generic kmalloc() is served by power-of-two-ish caches (kmalloc-8, -16, -32, -64, -96, -128, -192, … -8k); bigger requests go straight to the buddy allocator.

slabtop -o -s c | head -15         # biggest caches by size
#  OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME
# 9.8M  9.7M   99%    0.19K  468k       21    1.8G    dentry
# 2.1M  2.1M   99%    1.06K   70k       30    2.2G    ext4_inode_cache
grep -E 'Slab|SReclaimable|SUnreclaim' /proc/meminfo
ls /sys/kernel/slab/                # per-cache tunables and stats

War Story (illustrative)

Illustrative scenario — a composite of a commonly reported pattern.

A fleet of API servers showed MemAvailable steadily falling over weeks with no process growing. slabtop showed the dentry cache at 30GB. The service checked for the existence of a per-request temp file with stat() on a random path that almost never existed — and each miss created a negative dentry ("this name doesn't exist"), cached forever as long as there was no memory pressure. Dentries are SReclaimable, so the kernel would eventually have freed them, but the monitoring alerted on MemFree and the reclaim storms when pressure finally arrived caused latency spikes. The fix was to stop probing random paths; the monitoring was changed to alert on MemAvailable and PSI instead.

Slab objects are charged to the allocating task's memory cgroup, so a container that creates millions of sockets or files is billed for the kernel memory too (kernel and slab lines in memory.stat).


glibc ptmalloc2

The default Linux malloc (used by most C programs). Based on Doug Lea's allocator. Key properties:

  • Arenas: multiple independent heaps to reduce lock contention in multi-threaded programs. Thread tries to use its assigned arena; falls back to a new one if locked.
  • Bins: free lists organized by size class. Fastbins for small allocations (no coalescing). Unsorted bin for recently freed blocks. Small bins (exact sizes). Large bins (size ranges).
  • Thread cache (tcache): per-thread cache of recently freed blocks for common sizes. Allows malloc/free without any lock in the common case.

Problems:

  • Multiple arenas fragment virtual memory
  • Arena lock contention at high thread counts
  • Poor performance for workloads with many concurrent allocations/frees

The numbers behind the design (glibc 2.26+ on 64-bit)

StructureSizes servedCount / capacityBehaviour
tcachechunks up to 1032 bytes64 bins × 7 entries each, per threadNo locks, LIFO, no coalescing
fastbinsup to 128 bytes (default global_max_fast), max 16010 bins, per arenaSingly linked LIFO, no coalescing until consolidation
unsorted binany1 per arenaFreed chunks land here first; sorted into small/large bins lazily on the next malloc
small bins< 1024 bytes62 bins, exact 16-byte spacingDoubly linked FIFO, coalesced
large bins≥ 1024 bytes63 bins, logarithmic rangesSorted by size, best-fit
top chunk—1 per arenaWilderness at the end of the heap; split when bins are empty
arenas—up to 8 × cores (64-bit)Main arena uses brk; others use 64MB mmap'd heaps
flowchart TD
    M["malloc(n)"] --> T{"tcache bin for size has a chunk?"}
    T -->|"yes"| R1["Return it (no lock, ~15ns)"]
    T -->|"no"| BIG{"n above mmap threshold?"}
    BIG -->|"yes"| MM["mmap a dedicated chunk"]
    BIG -->|"no"| L["Lock this thread's arena"]
    L --> FB{"fastbin or small bin exact fit?"}
    FB -->|"yes"| R2["Return chunk, refill tcache from same bin"]
    FB -->|"no"| UB["Process unsorted bin: return exact fit, sort others into bins"]
    UB --> LB{"large bin best fit?"}
    LB -->|"yes"| R3["Split, remainder to unsorted bin"]
    LB -->|"no"| TOP{"Top chunk big enough?"}
    TOP -->|"yes"| R4["Split top chunk"]
    TOP -->|"no"| GROW["Grow heap: brk (main arena) or new mmap heap"]

The arena explosion

Each secondary arena reserves a 64MB region of address space (HEAP_MAX_SIZE). A JVM or Ruby process with 200 threads can end up with 8 × 32 cores = 256 arenas: tens of GB of VSZ and, more importantly, hundreds of partially used heaps that each hold onto freed memory. This is the famous "64MB blocks in pmap" signature:

pmap -x <pid> | awk '$2 >= 65000 && $2 <= 65536' | wc -l   # count ~64MB arena heaps

The standard mitigation is to cap arenas:

export MALLOC_ARENA_MAX=2          # Heroku's documented recommendation for Ruby apps
# or, glibc 2.26+:
export GLIBC_TUNABLES=glibc.malloc.arena_max=2:glibc.malloc.trim_threshold=131072

This trades some lock contention for much less fragmentation. Heroku documents MALLOC_ARENA_MAX=2 for Ruby, and Hadoop/YARN's docs have long recommended MALLOC_ARENA_MAX=4 for JVM containers for exactly this reason.

Giving memory back: malloc_trim

#include <malloc.h>
malloc_trim(0);   // return free memory to the OS: shrinks the brk top AND
                  // madvise(MADV_DONTNEED)s free pages inside every arena (glibc 2.8+)
malloc_stats();   // per-arena summary to stderr
malloc_info(0, stdout);   // XML: per-arena, per-bin detail
struct mallinfo2 mi = mallinfo2();   // glibc 2.33+: 64-bit-safe counters (mallinfo() overflows past 2GB)

Calling malloc_trim(0) after a known spike (end of a batch, after cache eviction) can drop RSS dramatically on glibc. It walks all arenas under their locks, so don't call it on a hot path.

$ ./app   # with malloc_stats() at exit
Arena 0:
system bytes     =  135168000
in use bytes     =   12582912
Arena 1:
system bytes     =   67108864
in use bytes     =    1048576
Total (incl. mmap):
system bytes     =  202276864      ← RSS-ish: what glibc holds from the OS
in use bytes     =   13631488      ← what the app is actually using: 7% utilisation
max mmap regions =         12

tcmalloc (Thread-Caching Malloc)

Google's allocator used in Chrome, production services. Designed for high-concurrency workloads.

Architecture:

Thread-local cache: per-thread pool of small objects (< 256KB)
  ↕ (refill/return in batches)
Central cache: per-size-class lists, one lock per size class
  ↕
Page heap: manages spans (runs of pages) from the OS

Key advantages:

  • Thread-local cache makes most small allocations/frees lock-free
  • Batch transfers between thread cache and central cache reduce lock frequency
  • Low fragmentation — tight size classes minimize internal fragmentation

Performance: 2–8x faster than ptmalloc2 for highly concurrent workloads.

(Treat that range as "in allocation-heavy multi-threaded microbenchmarks". Modern glibc with tcache has closed much of the single-threaded gap; real application gains are usually single-digit to low-double-digit percent in CPU, and the bigger win is often memory.)

Two tcmallocs

There are two codebases with this name, and their behaviour differs:

gperftools tcmallocgoogle/tcmalloc (2020+)
Front-end cachePer-threadPer-CPU (using Linux restartable sequences, rseq)
Huge pagesObliviousHugepage-aware ("Temeraire") back end
ProfilingHEAPPROFILE, pprofBuilt-in sampled profiling exposed to Google's profilers
Buildautotools, packaged in distros (libgoogle-perftools)Bazel/CMake; less often packaged
Used byMany open-source projects via distro packagesGoogle's fleet, MongoDB 8.0+

Per-CPU caches matter at high thread counts: with per-thread caches, 2,000 threads × a few MB of cache each is gigabytes of idle cached memory; with per-CPU caches, the number of caches is bounded by core count, and each one stays hot. rseq lets a thread operate on "its" CPU's cache without locks, aborting and retrying if it gets preempted or migrated mid-operation.

The hugepage-aware back end packs allocations so that whole 2MB regions are either densely used or entirely free (and can be released), which makes THP pay off instead of bloating RSS. Google's OSDI 2021 paper "Beyond malloc efficiency to fleet efficiency: a hugepage-aware memory allocator" reported fleet-wide throughput gains of several percent and reduced memory use — at Google's scale, a large number of machines' worth of savings.


jemalloc

Originally written by Jason Evans for FreeBSD (it became FreeBSD's libc allocator in 2005–2006), then developed for years at Facebook/Meta, where it backs most C++ services; Firefox uses a long-diverged fork (mozjemalloc), and Redis bundles it as its default allocator on Linux. Designed for low fragmentation and memory profiling.

Key features:

  • Size classes: 232 size classes with careful spacing to minimize internal fragmentation
  • Arenas: multiple arenas (default: 4× CPU count). Thread bound to an arena.
  • Thread-local caching: per-thread cache, similar to tcmalloc
  • Extent management: tracks memory at the "extent" (multi-page) level for better fragmentation control
  • Built-in profiling: heap profiling with MALLOC_CONF=prof:true — samples allocations and produces flame graphs

The size classes are spaced so that there are four classes per doubling (e.g. 64, 80, 96, 112, 128, 160, 192, 224, 256, …): worst-case internal fragmentation is bounded at about 20%, versus up to 50% for pure power-of-two classes. Small objects (up to 14KB with 4KB pages) live in slabs — runs of pages holding objects of one size class, with an allocation bitmap — and larger ones get their own extents.

Purging: how jemalloc gives memory back

This is the most important thing to understand about jemalloc in production. Free pages go through two stages before returning to the OS, each with a time-based decay:

stateDiagram-v2
    [*] --> Active: allocated
    Active --> Dirty: freed (still resident)
    Dirty --> Muzzy: after dirty_decay_ms (MADV_FREE)
    Muzzy --> Retained: after muzzy_decay_ms (MADV_DONTNEED)
    Dirty --> Active: reused (no page fault)
    Muzzy --> Active: reused
    Retained --> Active: reused (new page faults)
  • dirty pages are free but still resident — cheap to reuse.
  • muzzy pages have been MADV_FREEd — the kernel may reclaim them under pressure; RSS doesn't drop until it does.
  • Defaults: dirty_decay_ms:10000, muzzy_decay_ms:0 (since 5.2 on Linux) — free memory drifts back to the OS over about 10 seconds.
  • background_thread:true does the purging on a background thread rather than inline in malloc/free calls, which smooths latency.
# Common production MALLOC_CONF
export MALLOC_CONF="background_thread:true,dirty_decay_ms:5000,narenas:8,metadata_thp:auto"

# Print statistics at exit
MALLOC_CONF=stats_print:true ./app 2>&1 | grep -E 'Allocated|active|resident|mapped|retained'
# Allocated: 1503912488, active: 1576431616, metadata: 32418112, resident: 1652400128,
#   mapped: 1789136896, retained: 412090368

Read those as: allocated (bytes the app holds) ≤ active (in pages with live objects) ≤ resident (RSS attributable to jemalloc) ≤ mapped. resident / allocated is your fragmentation-plus-cache overhead — here about 1.10, which is very healthy.

In code, the same numbers are available via mallctl:

#include <jemalloc/jemalloc.h>
uint64_t epoch = 1; size_t sz = sizeof(epoch);
mallctl("epoch", &epoch, &sz, &epoch, sz);           // refresh cached stats
size_t allocated, resident; sz = sizeof(size_t);
mallctl("stats.allocated", &allocated, &sz, NULL, 0);
mallctl("stats.resident",  &resident,  &sz, NULL, 0);
mallctl("arena.4096.purge", NULL, NULL, NULL, 0);      // 4096 = MALLCTL_ARENAS_ALL: purge now

Why Redis uses jemalloc: Redis's memory usage must be predictable and reportable. jemalloc's fragmentation behavior is more consistent than ptmalloc2, and its reporting is more accurate.

It also makes Redis's activedefrag possible (above), and INFO memory reports allocator_allocated, allocator_active, allocator_resident straight from jemalloc's stats.

Project status

In 2025 jemalloc's original author archived the upstream GitHub repository and published a retrospective ("jemalloc Postmortem"). The allocator remains in wide use and large users maintain their own branches, but check the current maintenance status before choosing it for a new project; tcmalloc and mimalloc are actively developed alternatives.


mimalloc

Microsoft Research's allocator (Daan Leijen et al., 2019). Small (~10k lines), very fast, and with an unusual design centred on free-list sharding:

  • Memory is divided into pages (64KB for small objects) each holding one size class, grouped into segments.
  • Each page has three free lists instead of one per size class: a free list for allocation, a local-free list that the owning thread frees into, and a thread-free list that other threads free into atomically. Allocation pops from the free list with no atomics; when it's empty, the other two are swapped in in bulk.
  • Because free lists are per page and short, allocations from the same page are close together in memory — good locality — and a page whose objects are all freed can be returned to the OS as a unit.
  • A secure mode adds guard pages, encrypted free-list pointers and randomised allocation order to blunt heap exploits, for ~10% cost.

mimalloc is used inside Microsoft services, is the default in some language runtimes (Koka; CPython 3.13's free-threaded "no-GIL" build uses it internally), and is popular as a drop-in in Rust (mimalloc crate) and C# native code. On many allocation-heavy benchmarks it is as fast or faster than jemalloc and tcmalloc with similar or lower memory use; on some workloads with lots of cross-thread frees it can retain more memory.

// Rust: switch the global allocator in two lines
use mimalloc::MiMalloc;
#[global_allocator]
static GLOBAL: MiMalloc = MiMalloc;
// or tikv_jemallocator::Jemalloc, or tcmalloc via the `tcmalloc` crate

Choosing an Allocator

AllocatorBest for
ptmalloc2 (glibc)General purpose, single-threaded or low-contention
tcmallocHigh-concurrency servers, latency-sensitive (Google stack)
jemallocServers where memory fragmentation matters (databases, caches)
mimallocGeneral-purpose alternative, very fast for small allocations

A more detailed comparison:

glibc ptmalloc2jemalloctcmalloc (google)mimalloc
Fast pathper-thread tcache (7/bin)per-thread tcacheper-CPU cache (rseq)per-thread page free lists
Contention at 64+ threadsArena locks; tune arena_maxLowVery lowLow
Returning memory to OSPoor (top-of-heap trim, malloc_trim)Good, time-decay purgingGood, hugepage-awareGood, per-page
Fragmentation in long-running serversOften highLowLowLow
Huge page friendlinessPoorOK (metadata_thp, tuned decay)ExcellentOK
Heap profilingNone built in (use external tools)Excellent (prof:true, jeprof)Excellent (sampled)Basic stats
Drop-in via LD_PRELOAD—YesYes (gperftools easiest)Yes
Typical RSS change switching from glibc—−10% to −50% on fragmenting workloadssimilarsimilar

When to switch

Switch from glibc when you see any of:

  • RSS climbing over days while the application's own accounting of live objects is flat
  • perf shows significant time in malloc/free/_int_malloc/__lll_lock_wait under concurrency
  • Many ~64MB anonymous mappings in pmap (arena explosion)
  • You need heap profiling in production

Don't bother when the program is short-lived, single-threaded, or allocates little — and be careful in programs that mix allocators (memory malloced by one and freed by another is a crash), use dlopened plugins with static allocators, or rely on glibc-specific functions like malloc_trim.

# Try it with zero code changes
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2 ./server
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libtcmalloc_minimal.so.4 ./server
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libmimalloc.so ./server

# Docker: install the package and set ENV LD_PRELOAD=... in the Dockerfile
# Permanently: link with -ljemalloc (or -ltcmalloc), which takes precedence over libc's malloc

Production stories

  • Ruby on Rails memory bloat. Ruby's many threads (Puma, Sidekiq) and glibc's arenas produced RSS growth that looked like leaks. Mike Perham (Sidekiq) and others documented in 2018 that running with jemalloc or MALLOC_ARENA_MAX=2 cut memory use by roughly half on typical Sidekiq workloads; Heroku made MALLOC_ARENA_MAX=2 part of its Ruby guidance, and jemalloc-enabled Ruby builds became common.
  • Rust dropped jemalloc as the default. Until Rust 1.32 (2019), binaries bundled jemalloc. It was removed in favour of the system allocator to cut binary size and platform issues; teams that need it opt back in via #[global_allocator] (as TiKV and many others do).
  • Redis and jemalloc. Redis switched its default to jemalloc on Linux in 2.4 (2011) after fragmentation issues with glibc, and later built active defragmentation on jemalloc's internals.
  • MongoDB and tcmalloc. MongoDB has used tcmalloc for years; 8.0 moved to the newer per-CPU, hugepage-aware tcmalloc and changed its THP recommendation from "disable" to "enable" as a result.

Arena, Region and Pool Allocators

General-purpose allocators pay for generality: headers, size lookups, thread safety, and per-object free. When you know the lifetime pattern of your allocations, you can do dramatically better with a purpose-built allocator layered on top.

Bump / arena (region) allocators

Allocate by incrementing a pointer; free everything at once by resetting it. No per-object headers, no per-object free, O(1) and ~1–2 ns per allocation.

typedef struct { char *base, *ptr, *end; } Arena;

void *arena_alloc(Arena *a, size_t n) {
    n = (n + 15) & ~(size_t)15;                 // 16-byte alignment
    if (a->ptr + n > a->end) return NULL;        // or chain a new block
    void *p = a->ptr; a->ptr += n; return p;
}
void arena_reset(Arena *a) { a->ptr = a->base; } // "free" every object at once
// Rust: bumpalo — everything allocated in `bump` is freed when it's dropped
let bump = bumpalo::Bump::new();
for tok in tokens {
    let node: &mut Node = bump.alloc(Node::from(tok));   // pointer bump, no malloc call
    tree.push(node);
}
handle(&tree);
// `bump` dropped at end of scope: one deallocation for the whole request

Perfect for per-request or per-frame data: parsers and compilers (AST nodes), HTTP request handling, game frame allocations, query execution in databases (PostgreSQL's memory contexts, Apache APR pools, and nginx's ngx_pool_t are all region allocators — nginx frees a request's whole pool when the request ends, which is a big reason it doesn't leak). The risk: anything that must outlive the region needs to be copied out, and one long-lived reference keeps the whole region alive.

Go experimented with this as GOEXPERIMENT=arenas (Go 1.20); the proposal is on hold because it is hard to make memory-safe in a GC'd language — a pointer into a freed arena must fault rather than silently read reused memory.

Pool (free-list / object pool) allocators

Fixed-size objects recycled through a free list — the user-space version of the slab idea. Good for objects with expensive initialisation or very high churn: connection objects, buffers, message structs.

// Go: sync.Pool caches reusable objects per P and lets the GC clear them
var bufPool = sync.Pool{
    New: func() any { return new(bytes.Buffer) },
}

func render(w io.Writer, v any) {
    buf := bufPool.Get().(*bytes.Buffer)
    buf.Reset()
    defer bufPool.Put(buf)
    _ = json.NewEncoder(buf).Encode(v)
    w.Write(buf.Bytes())
}

Pools can leak and lie

A pool that never shrinks is a leak with good intentions: one burst of 10,000 concurrent requests leaves 10,000 buffers parked forever. Pools also hide allocations from profilers and defeat tools like ASan (use-after-return-to-pool isn't detected). sync.Pool avoids the first problem by letting the GC drop pooled items over two GC cycles; hand-rolled pools need a cap. And pooling large buffers of varied size can make fragmentation worse — bucket them by size class.

PatternAlloc costFree costBest forWatch out for
General malloc10–50 ns10–50 nsAnythingFragmentation, contention
Bump/arena1–2 ns~0 (bulk reset)Same-lifetime batchesLifetime bugs, retention
Fixed-size pool~5 ns~5 nsHot fixed-size objectsUnbounded growth
Stack allocation~0~0Small, scoped dataStack size limits

Garbage Collected Languages

GC languages (Java, Go, Python) manage memory automatically, but still rely on an underlying allocator and have their own allocation strategies.

Go's allocator: Inspired by tcmalloc. Size classes from 8B to 32KB. Objects ≤ 32KB go to per-P (processor) caches; larger objects go directly to the heap. GC uses tri-color mark-and-sweep with write barriers.

JVM G1GC: Heap divided into equal-sized regions. Young regions collect frequently (minor GC). Old regions collect when full (major GC). Aims for predictable pause times.

Python: Reference counting for immediate deallocation + cycle detector for reference cycles. CPython has a small object allocator (pymalloc) for objects ≤ 512 bytes organized in arenas/pools/blocks.

Go and the OS

Go's allocator layers are mcache (per P, no locks) → mcentral (per size class, ~68 classes) → mheap (spans of 8KB pages, obtained from the OS in 64MB heap arenas). Objects that don't escape the function are put on the stack by the compiler and never touch the allocator at all (go build -gcflags=-m shows escape decisions).

The GC decides when to collect, and a separate scavenger decides when to give free pages back to the OS:

KnobEffect
GOGC=100 (default)Next GC when the heap grows 100% over live data after the last GC — so heap ≈ 2× live
GOMEMLIMIT (Go 1.19+)Soft total memory limit; GC runs more often as the limit approaches. Set ~90% of container limit
GOGC=off + GOMEMLIMITCollect only when nearing the limit — good for batch jobs with a known budget
ScavengerReturns pages with MADV_DONTNEED in the background, targeting retained memory close to the heap goal
var m runtime.MemStats
runtime.ReadMemStats(&m)
fmt.Printf("HeapAlloc=%d HeapInuse=%d HeapIdle=%d HeapReleased=%d Sys=%d NumGC=%d\n",
    m.HeapAlloc, m.HeapInuse, m.HeapIdle, m.HeapReleased, m.Sys, m.NumGC)
// RSS ≈ HeapInuse + (HeapIdle − HeapReleased) + stacks + runtime metadata

Why Go RSS sometimes didn't drop (MADV_FREE)

Go 1.12–1.15 returned memory with MADV_FREE, which lets the kernel reclaim lazily — so RSS stayed high until there was memory pressure, and container dashboards showed "leaks" that weren't. Go 1.16 switched back to MADV_DONTNEED by default (RSS drops immediately). The same effect exists in jemalloc's muzzy pages and in any allocator using MADV_FREE: RSS is not the same as "memory that can't be reclaimed".

JVM: heap vs RSS

The single most common JVM memory surprise: -Xmx4g and a container OOM-killed at 6GB. The heap is only one of many consumers:

RegionTypical sizeControlled by
Java heap-Xmx-Xmx, -XX:MaxRAMPercentage
Metaspace (class metadata)50–500 MB-XX:MaxMetaspaceSize
Code cache (JIT output)50–250 MB-XX:ReservedCodeCacheSize
Thread stacks1 MB reserved per thread (-Xss), less residentthread count
GC data structures2–10% of heap (G1 remembered sets, card tables)collector choice
Direct / NIO bufferscan be GBs (Netty, Kafka clients)-XX:MaxDirectMemorySize
Native allocations via glibc (JNI, zlib, SSL)variesMALLOC_ARENA_MAX, alternative allocator
Arena fragmentation in glibccan be hundreds of MBMALLOC_ARENA_MAX=2..4, jemalloc
java -XX:NativeMemoryTracking=summary -jar app.jar &
jcmd <pid> VM.native_memory summary
# Total: reserved=7834MB, committed=5921MB
# -  Java Heap (reserved=4096MB, committed=4096MB)
# -  Class     (reserved=1101MB, committed=161MB)
# -  Thread    (reserved=412MB,  committed=412MB)    ← 400 threads
# -  Code      (reserved=250MB,  committed=98MB)
# -  GC        (reserved=210MB,  committed=210MB)
# -  Internal / Other (direct buffers) ...

If NMT's committed total is well below RSS, the gap is usually native allocations outside the JVM's view — glibc arenas from JNI libraries — which is where MALLOC_ARENA_MAX or LD_PRELOADing jemalloc earns its keep (and jemalloc's profiler can then tell you which native call sites allocate).

Python

pymalloc handles objects ≤ 512 bytes in pools (one size class each) inside arenas (256KB before Python 3.10, 1MB since), and returns an arena to the OS only when every object in it is freed — so a few surviving objects can pin whole arenas, the same pinned-page problem as glibc. Larger objects go to the system malloc. PYTHONMALLOC=malloc bypasses pymalloc (useful with valgrind or to try jemalloc); tracemalloc is the built-in allocation tracker:

import tracemalloc
tracemalloc.start(25)
# ... run workload ...
snap = tracemalloc.take_snapshot()
for stat in snap.statistics("lineno")[:10]:
    print(stat)       # file:line  size=48.2 MiB  count=402113  average=126 B

Observability

# Linux: per-process memory breakdown
cat /proc/<pid>/status        # VmRSS (resident), VmSize (virtual)
pmap -x <pid>                 # per-mapping breakdown

# Heap profiling with jemalloc
MALLOC_CONF=prof:true,prof_final:true ./app
jeprof --pdf app heap.prof > heap.pdf

# Heap profiling with tcmalloc (gperftools)
LD_PRELOAD=/usr/lib/libtcmalloc.so HEAPPROFILE=/tmp/heap ./app
pprof --pdf ./app /tmp/heap.0001.heap > heap.pdf

# Valgrind: memory error detection (slow but thorough)
valgrind --leak-check=full ./app

# AddressSanitizer: fast memory error detection (~2x slowdown)
clang -fsanitize=address -g ./app.c -o app && ./app

(Corrections to the original snippet: heap profiling in gperftools lives in libtcmalloc.so — libprofiler.so is the CPU profiler; jemalloc must be built with --enable-prof for prof:true to work, and it writes files named jeprof.<pid>.<seq>.f.heap at exit, so the jeprof command takes that filename.)

Leak, fragmentation, or cache? Decide first

"Memory keeps growing" has three different causes needing three different tools. Work out which one you have before reaching for valgrind:

flowchart TD
    A["RSS grows over time"] --> B{"Does the app's own live-object count grow too? (heap profile, GC live bytes)"}
    B -->|"yes"| C{"Are the objects reachable?"}
    C -->|"yes, intentionally"| CACHE["Unbounded cache or queue: add bounds or eviction"]
    C -->|"no (C/C++) or unexpectedly (GC)"| LEAK["Leak: heaptrack, jemalloc prof, LSan, pprof diff"]
    B -->|"no, flat"| D{"Allocator resident much greater than allocated?"}
    D -->|"yes"| FRAG["Fragmentation or retention: arena_max, malloc_trim, switch allocator, decay settings"]
    D -->|"no"| E["Outside the heap: thread stacks, mmaps, page cache, tmpfs, kernel memory - check smaps and memory.stat"]

The toolbox

ToolFindsOverheadUse in production?
Valgrind memcheckLeaks, invalid reads/writes, use of uninitialised memory20–50× slowerNo — tests only
Valgrind massif + ms_printHeap usage over time and peak contributors~20×No
AddressSanitizer / LeakSanitizerOverflows, use-after-free, leaks at exit~2× CPU, 2–3× memoryCanary or staging
heaptrackEvery allocation with stack traces; leaks, peak, temporary allocations~2–5×Short sessions on a canary
jemalloc profiling (prof:true,lg_prof_sample:19)Sampled live allocations by stack (every ~512KB)~1–5%Yes
tcmalloc / gperftools heap profilerSampled allocationsLowYes
bcc memleak (eBPF)Outstanding allocations by stack, for any process, no restartModerate (traces every malloc)Briefly
Go pprof heapinuse_space / alloc_space by stack~0 (sampled)Yes
JVM NMT, heap dumps, JFRNative categories, object graphs, allocation samplingLow–moderateYes (JFR)
malloc_stats / malloc_infoglibc arena utilisation~0Yes
# heaptrack: record and analyse
heaptrack ./app            # writes heaptrack.app.<pid>.zst
heaptrack --analyze heaptrack.app.12345.zst | head -60   # or heaptrack_gui
# "PEAK MEMORY CONSUMERS", "MOST TEMPORARY ALLOCATIONS", "MEMORY LEAKED" sections

# jemalloc: profile a running service, dumping every 1GB of allocation growth
export MALLOC_CONF="prof:true,prof_active:true,lg_prof_sample:19,lg_prof_interval:30,prof_prefix:/tmp/jeprof"
./server &
# later: compare two dumps to see what grew
jeprof --text --base=/tmp/jeprof.1234.3.i3.heap ./server /tmp/jeprof.1234.9.i9.heap | head -20

# eBPF: who is holding unfreed memory right now? (bcc-tools)
/usr/share/bcc/tools/memleak -p <pid> --top 10 30
# [12:04:31] Top 10 stacks with outstanding allocations:
#   104857600 bytes in 25600 allocations from stack
#       malloc+0x...  lru_insert+0x52 [server]  handle_set+0x1a0 [server] ...

# Go
curl -s localhost:6060/debug/pprof/heap > h1.pb.gz; sleep 600
curl -s localhost:6060/debug/pprof/heap > h2.pb.gz
go tool pprof -top -diff_base=h1.pb.gz h2.pb.gz

Diff, don't stare

A single heap profile shows what's big; a leak shows up as what grows. Take two profiles some time apart under steady load and diff them (--base, -diff_base). The top of the diff is almost always the leak.

For deeper eBPF-based allocation tracing, see eBPF.


Real Numbers

QuantityValue
Fast-path malloc/free (tcache, jemalloc, tcmalloc, mimalloc)~10–20 ns
Contended glibc arena lock path100 ns – several µs
brk/mmap syscall~0.2–1 µs + page faults on first touch
Minor page fault on fresh anonymous memory~0.5–2 µs per 4KB
glibc minimum chunk / overhead per allocation (64-bit)32 bytes / 8 bytes
glibc mmap threshold128 KiB (dynamic up to 32 MiB)
glibc max arenas (64-bit)8 × cores, 64 MiB reserved each
glibc tcache64 bins × 7 chunks, ≤ 1032 bytes
jemalloc size classes / spacing~232 / 4 per doubling (≤ ~20% internal waste)
jemalloc default dirty_decay_ms10 s
Go size classes / max small object~68 / 32 KiB
pymalloc small object limit512 bytes
Linux buddy max order10 (4 MiB)
Typical RSS reduction switching glibc → jemalloc on fragmenting servers10–50%

Interview Questions

"What is heap fragmentation? How do allocators deal with it?" External fragmentation: free memory is split into chunks too small to satisfy large allocations. Internal fragmentation: allocated blocks are larger than requested (due to size rounding). Allocators combat fragmentation with size classes (allocations rounded to one of ~50 sizes, reducing internal fragmentation), coalescing (merging adjacent free blocks to reduce external fragmentation), and thread-local caches (reducing the spread of allocations across the heap).

"Why would you use jemalloc instead of the default glibc malloc?" jemalloc has better fragmentation characteristics for long-running servers — the glibc allocator tends to accumulate fragmentation over time, causing RSS to grow unboundedly even when the application's logical heap size is stable. jemalloc's more careful size class design and arena management keep fragmentation bounded. Also: built-in heap profiling.

"What happens when you call malloc() at a system level?" For small allocations: check thread-local cache (no lock). If empty, refill from a size-class free list (lock per size class). If empty, allocate pages from the OS via mmap(MAP_ANONYMOUS). For large allocations (>128KB typically): call mmap directly and return the mapping. free() returns the block to the thread cache or to the size-class free list. (In glibc specifically, the main arena grows with brk and secondary arenas with mmap; and nothing physical is allocated until the pages are first touched and page-fault.)

"Why doesn't RSS go down after I free memory?" free returns memory to the allocator, not the OS. glibc can only shrink its main heap from the top and otherwise keeps freed memory for reuse; other allocators return pages on a timer (jemalloc decay, Go scavenger) or lazily via MADV_FREE, which leaves RSS high until the kernel needs the memory. Fragmentation — a few live objects per page — can prevent return entirely.

"How does the Linux kernel allocate memory for its own objects?" Pages come from the buddy allocator (power-of-two blocks of pages, buddies found by XOR, coalesced on free). Small, frequent objects come from SLUB caches — per-type slabs of fixed-size objects with a per-CPU fast path — and kmalloc uses generic size-class caches.

"A JVM with -Xmx4g uses 6GB of RSS. Where's the rest?" Metaspace, code cache, thread stacks, GC structures, direct buffers, and native allocations through glibc (plus arena fragmentation). Measure with Native Memory Tracking; if NMT's total is below RSS, look at glibc arenas (MALLOC_ARENA_MAX) or switch the native allocator.

"When would you write your own allocator?" When allocation lifetimes are structured: per-request or per-frame data (arena — allocate by bumping a pointer, free everything at once), or many identical hot objects (pool). Not to beat jemalloc at general-purpose allocation.