Memory Allocators
Every time you call malloc() in C, new in C++, or allocate an object in a garbage-collected language, a memory allocator decides where that memory comes from. The allocator manages the heap: tracking free regions, splitting and coalescing blocks, and deciding when to ask the kernel for more memory. Poor allocator design causes fragmentation, lock contention, and mysterious performance cliffs. Good allocator design — like jemalloc in Firefox and Meta's servers, or tcmalloc in Chrome and Google — is a meaningful competitive advantage.
The allocator sits in a gap that most engineers never look at: below your language, above the kernel. It is responsible for some of the most confusing production behaviour you will see — a service whose RSS grows for days and never shrinks even though its "live data" is flat, a multi-threaded program that gets slower when you add cores, a container OOM-killed with a heap well under its limit, or memory that drops by 40% after a one-line LD_PRELOAD change. This page explains how allocators work from first principles, how the major production allocators differ, and how to diagnose allocator problems with real tools.
Prerequisites
Read Virtual Memory first — especially demand paging, mmap, huge pages, and RSS vs PSS. Allocators are built entirely on those mechanisms.
Where the Allocator Sits
There are two allocators in every Linux process's life: the user-space allocator (malloc, part of libc or a replacement library) and the kernel's allocators (the buddy page allocator and the slab allocator). They solve the same problem at different granularities.
flowchart TD
APP["Application: malloc(48), new Foo, make([]byte, n)"] --> UA["User-space allocator (ptmalloc, jemalloc, tcmalloc, mimalloc, Go runtime)"]
UA -->|"brk / mmap / madvise"| SYS["System calls"]
SYS --> VMA["Kernel VM: VMAs and page tables"]
VMA -->|"page fault on first touch"| BUDDY["Buddy page allocator (4KB..4MB blocks)"]
KOBJ["Kernel objects: inodes, sockets, task_struct"] --> SLUB["SLUB slab allocator"]
SLUB --> BUDDY
BUDDY --> RAM["Physical frames"]
Two consequences matter constantly in practice:
- The user-space allocator asks the kernel for big chunks and carves them up. Syscalls cost hundreds of nanoseconds; a good
malloccosts 10–20 ns. So almost allmalloc/freecalls never reach the kernel. - Freeing memory in your program is not the same as returning it to the OS.
free()gives a block back to the allocator. Whether the allocator gives the underlying pages back to the kernel (munmap,madvise(MADV_DONTNEED),brkshrink) is a separate policy decision — and the source of most "my RSS never goes down" confusion.
The Heap and brk()/mmap()
The heap is a region of virtual memory that grows upward as the program allocates more. The kernel exposes two ways to get more memory:
brk(addr)— moves the end of the heap segment toaddr. Cheap, contiguous. Only works for growing/shrinking the heap monotonically.mmap(MAP_ANONYMOUS)— maps a new anonymous page anywhere in the address space. More flexible; used for large allocations.
malloc manages a pool of memory obtained from these syscalls. For small allocations, it carves up a larger slab. For large allocations (typically > 128KB), it directly calls mmap and returns the mapping.
You can watch this happen:
// heap_demo.c — gcc -O0 heap_demo.c -o heap_demo && strace -e trace=brk,mmap,munmap ./heap_demo
#include <stdlib.h>
#include <stdio.h>
int main(void) {
void *small = malloc(100); // served from the brk heap
void *big = malloc(1 << 20); // 1 MiB > threshold: its own mmap
printf("small=%p big=%p\n", small, big);
free(big); // munmap immediately
free(small); // stays in the heap for reuse
return 0;
}
brk(NULL) = 0x5583f1a6a000
brk(0x5583f1a8b000) = 0x5583f1a8b000 ← heap grows by 132KB for a 100-byte malloc
mmap(NULL, 1052672, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9a2c5ff000
small=0x5583f1a6a2a0 big=0x7f9a2c5ff010
munmap(0x7f9a2c5ff000, 1052672) = 0
Notice the first brk grows the heap by 132KB (the 128KB M_TOP_PAD default plus rounding) even though we asked for 100 bytes — amortising future syscalls. The 1 MiB request gets its own mapping, 4KB bigger than requested to hold the chunk header and round to a page.
glibc's thresholds
| Tunable | Default (64-bit) | Meaning |
|---|---|---|
M_MMAP_THRESHOLD | 128 KiB, dynamic up to 32 MiB | Requests at least this big go straight to mmap |
M_TRIM_THRESHOLD | 128 KiB (doubles with the dynamic mmap threshold) | When free space at the top of the brk heap exceeds this, shrink the heap with brk |
M_TOP_PAD | 128 KiB | Extra slack added when growing the heap |
M_ARENA_MAX | 8 × CPU cores | Maximum number of arenas |
M_MMAP_MAX | 65536 | Maximum simultaneous mmap'd chunks |
The dynamic threshold is a subtle trap: when you free() an mmap'd chunk, glibc raises the threshold to that chunk's size, on the theory that you'll allocate similar sizes again and they're better served from the heap. A program that allocates and frees a few 10MB buffers early will have all later 10MB buffers served from arenas — where they can fragment and may never be returned. Setting the threshold explicitly (mallopt(M_MMAP_THRESHOLD, ...) or GLIBC_TUNABLES=glibc.malloc.mmap_threshold=131072) disables the dynamic adjustment.
brk heap | mmap per allocation | |
|---|---|---|
| Syscall per alloc | No (amortised) | Yes — mmap + munmap each, plus page faults |
Returned to OS on free | Only from the top, past the trim threshold | Immediately |
| Fragmentation | Possible | None (but 4KB-granular internal waste) |
| Page-fault cost | Once per page, reused afterwards | Every time: fresh zeroed pages for every allocation |
How a Simple malloc Works
A naive malloc maintains a free list — a linked list of free memory blocks, each containing a header with size and a next pointer.
Heap memory:
[hdr| 16 bytes free |hdr| 32 bytes used |hdr| 64 bytes free ]
↑
free list: head → block1(16) → block3(64) → NULL
malloc(n) steps:
- Scan free list for a block ≥ n bytes (first-fit, best-fit, or worst-fit strategies)
- If found: split the block (return n bytes, keep remainder in free list)
- If not found: call
brk()/mmap()to get more memory from the kernel
free(ptr) steps:
- Mark the block as free (restore it to the free list)
- Attempt to coalesce adjacent free blocks into one larger block (prevents fragmentation)
How does free(ptr) know the size of the block, when you never told it? The header sits just before the pointer malloc returned. And how does it coalesce with the previous block in O(1)? Boundary tags: the size is stored at both ends of a free block (or, in glibc, the previous chunk's size is stored at the start of the next chunk while the previous chunk is free), so you can step backwards.
Here is glibc's real chunk layout on 64-bit:
+--------------------------------------------+
chunk -> | prev_size (only meaningful if prev is free) | 8 bytes
+--------------------------------------------+
| size of this chunk | A | M | P | 8 bytes; low 3 bits are flags
+--------------------------------------------+
mem -> | user data ... | ← pointer returned by malloc
| (when free: fd, bk pointers for the bin list)|
+--------------------------------------------+
next chunk | prev_size = this chunk's size (if free) |
P(PREV_INUSE): the previous chunk is allocated (so don't coalesce backwards)M(IS_MMAPPED): this chunk came from its ownmmapA(NON_MAIN_ARENA): this chunk belongs to a secondary arena
Sizes are multiples of 16, so the low bits are free for flags. The minimum chunk is 32 bytes. That gives the classic per-allocation overhead numbers: malloc(1) through malloc(24) use a 32-byte chunk; malloc(25) uses 48. If your program allocates 100 million 8-byte objects individually, you are using 3.2GB to store 800MB.
#include <malloc.h>
#include <stdio.h>
int main(void) {
for (size_t n = 1; n <= 64; n += 7)
printf("malloc(%2zu) -> usable %zu\n", n, malloc_usable_size(malloc(n)));
}
// malloc( 1) -> usable 24 malloc( 8) -> usable 24 malloc(22) -> usable 24
// malloc(29) -> usable 40 malloc(36) -> usable 40 malloc(50) -> usable 56 ...
Note the usable size of a 32-byte chunk is 24, not 16: an allocated chunk can use the next chunk's prev_size field, since that field is only meaningful when this chunk is free.
Fragmentation
Fragmentation is the primary enemy of a memory allocator.
External Fragmentation
Free memory exists but is split into small non-contiguous pieces — too small to satisfy a large allocation.
Heap: [free 16][used 32][free 16][used 32][free 16]
↑
Available free: 48 bytes total, but max contiguous = 16 bytes
malloc(32) fails — even though 48 bytes are free
Solutions: Coalescing adjacent free blocks; compaction (impossible if pointers exist); segregated free lists (keep separate pools per size class).
(In a real malloc the request doesn't actually fail — the allocator just grows the heap — so external fragmentation shows up as RSS that is much larger than live data, not as NULL returns.)
Internal Fragmentation
A block is allocated larger than requested (due to alignment or size rounding). The waste is inside allocated blocks.
malloc(9) on a 16-byte aligned allocator → 16-byte block allocated
7 bytes wasted inside the allocated block
Fragmentation in real servers: the "pinned top" problem
The textbook pictures above are about single allocations. In long-running servers the far more expensive effect is pages that can't be returned to the OS because one live object sits on them.
brk heap after a traffic spike (each box = one 4KB page):
[A][A][A][A][A][A][A][A][A][A][A][A] ← all allocated during the spike
after the spike, 95% of objects freed, but a few long-lived ones remain:
[A][ ][ ][ ][A][ ][ ][ ][ ][ ][ ][S]
↑ ↑ ↑
cache entry session one long-lived object at the TOP of the heap
- glibc's main arena can only shrink with
brkfrom the top. One live object near the top (S) pins everything below it. - Free pages in the middle could be returned with
madvise(MADV_DONTNEED), but glibc only does this inmalloc_trim(), which it rarely calls automatically. - Result: RSS stays at the spike's high-water mark forever, though live data is 5% of it.
This is why the same Ruby, Python or Java-native workload can use 2× the memory on glibc that it uses on jemalloc: jemalloc and tcmalloc track memory in page-sized runs and purge free pages back to the OS on a timer regardless of position.
Measure fragmentation as a ratio:
| Tool | Metric | Healthy |
|---|---|---|
Redis INFO memory | mem_fragmentation_ratio = RSS / used_memory | 1.0–1.5; > 1.5 investigate; < 1.0 means swapping |
jemalloc stats.resident / stats.allocated | resident vs requested | < 1.3 typical |
glibc malloc_info() | <system type="current"> vs sum of in-use | — |
Go runtime.MemStats | HeapInuse − HeapAlloc (span fragmentation), HeapIdle − HeapReleased (retained) | small |
Redis active defragmentation
Redis (with jemalloc) can fix fragmentation online: activedefrag yes scans keys, asks jemalloc whether the allocation sits in a poorly utilised run (je_get_defrag_hint), and if so reallocates and copies the value to a denser region, freeing the old run. That only works because Redis owns every pointer to its values — a general-purpose program can't move objects, which is exactly why C/C++ allocators can't compact.
The Buddy System
Used by the Linux kernel for page-level allocation. Memory is divided into blocks of power-of-two sizes. Each block has a "buddy" — the block of the same size immediately adjacent.
Allocation of size N (rounded up to next power of 2):
- Find the smallest free block ≥ N
- If it's larger than needed, split it into two buddies; place one in the free list
- Repeat until you have a block of the right size
Deallocation:
- Free the block
- Check if its buddy is also free
- If so, merge them into a block of double size
- Repeat until buddy is not free or max size reached
Splitting a 64-byte block to satisfy a 16-byte request:
64 → [32 | 32] → [16 | 16 | 32] → allocate left 16
Free list: 16(buddy), 32
Advantage: O(log N) allocation and deallocation, guaranteed coalescing. Disadvantage: Internal fragmentation — any allocation is rounded up to a power of 2. Requesting 33 bytes wastes 31 bytes (rounded to 64).
The elegance is in finding the buddy: for a block of size 2ᵏ at address a (aligned to 2ᵏ), its buddy is at a XOR 2ᵏ. No search needed.
The buddy allocator in Linux
In the kernel the unit is a page, and sizes are orders: order 0 = 1 page (4KB), order 1 = 2 pages, … order 10 = 1024 pages (4MB), the default maximum (MAX_PAGE_ORDER). Free lists are kept per NUMA node, per zone (DMA, DMA32, Normal), and per migrate type (movable, unmovable, reclaimable) so that unmovable kernel allocations don't get scattered across memory that could otherwise be compacted.
$ cat /proc/buddyinfo
Node 0, zone DMA 0 0 0 0 0 0 0 0 1 1 2
Node 0, zone DMA32 1543 882 517 310 178 97 45 21 9 4 310
Node 0, zone Normal 48211 20117 6012 1180 212 41 3 0 0 0 0
# order0 order1 ... order10
The Normal line above is a fragmented machine: tens of thousands of single free pages, but zero free blocks of order 7 or higher. A request for a 2MB huge page (order 9) must first run compaction — migrating movable pages to assemble a contiguous block — which is the root cause of the THP latency spikes described in Virtual Memory.
cat /sys/kernel/debug/extfrag/extfrag_index # near 1.0 = failures due to fragmentation
echo 1 > /proc/sys/vm/compact_memory # force a full compaction (careful in prod)
grep -E 'compact_(stall|fail|success)' /proc/vmstat
Per-CPU page lists (pcp) sit in front of the buddy allocator so that most order-0 allocations avoid the zone lock entirely.
Slab Allocator
The Linux kernel's primary allocator for kernel objects. Addresses the problem of frequently allocating and freeing objects of the same size (task structs, inodes, dentries, socket buffers).
Key insight: Instead of generic memory, maintain pools of pre-allocated, pre-initialized objects of each common type.
Slab cache for struct inode (size = 592 bytes):
Slab 1: [inode][inode][inode][inode] ... 8 inodes, 2 free
Slab 2: [inode][inode][inode][inode] ... 8 inodes, 8 free
Slab 3: empty (returned to page allocator)
alloc_inode(): grab from Slab 1's free list → O(1)
free_inode(): return to Slab 1's free list → O(1)
Advantages:
- No splitting/coalescing (fixed-size objects)
- Objects can retain partial initialization between uses (constructor called once, destructor skipped)
- Cache-friendly — objects of the same type are collocated
/proc/slabinfo shows all slab caches, their object sizes, and usage.
SLAB, SLUB and SLOB
The design above is Jeff Bonwick's slab allocator (SunOS 5.4, 1994), and Linux has had three implementations of the idea:
| Implementation | Status | Notes |
|---|---|---|
| SLAB | Removed in 6.8 (2024) | Original Linux port; per-CPU arrays plus complex queues per node |
| SLOB | Removed in 6.4 (2023) | Tiny-memory embedded variant |
| SLUB | Default since 2.6.23, now the only one | Simpler: per-CPU "active slab", free list threaded through the objects themselves, no per-slab metadata header, merging of caches with compatible sizes |
SLUB's fast path is a per-CPU pointer to a slab and a lockless cmpxchg on its free list — tens of nanoseconds. Generic kmalloc() is served by power-of-two-ish caches (kmalloc-8, -16, -32, -64, -96, -128, -192, … -8k); bigger requests go straight to the buddy allocator.
slabtop -o -s c | head -15 # biggest caches by size
# OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
# 9.8M 9.7M 99% 0.19K 468k 21 1.8G dentry
# 2.1M 2.1M 99% 1.06K 70k 30 2.2G ext4_inode_cache
grep -E 'Slab|SReclaimable|SUnreclaim' /proc/meminfo
ls /sys/kernel/slab/ # per-cache tunables and stats
War Story (illustrative)
Illustrative scenario — a composite of a commonly reported pattern.
A fleet of API servers showed MemAvailable steadily falling over weeks with no process growing. slabtop showed the dentry cache at 30GB. The service checked for the existence of a per-request temp file with stat() on a random path that almost never existed — and each miss created a negative dentry ("this name doesn't exist"), cached forever as long as there was no memory pressure. Dentries are SReclaimable, so the kernel would eventually have freed them, but the monitoring alerted on MemFree and the reclaim storms when pressure finally arrived caused latency spikes. The fix was to stop probing random paths; the monitoring was changed to alert on MemAvailable and PSI instead.
Slab objects are charged to the allocating task's memory cgroup, so a container that creates millions of sockets or files is billed for the kernel memory too (kernel and slab lines in memory.stat).
glibc ptmalloc2
The default Linux malloc (used by most C programs). Based on Doug Lea's allocator. Key properties:
- Arenas: multiple independent heaps to reduce lock contention in multi-threaded programs. Thread tries to use its assigned arena; falls back to a new one if locked.
- Bins: free lists organized by size class. Fastbins for small allocations (no coalescing). Unsorted bin for recently freed blocks. Small bins (exact sizes). Large bins (size ranges).
- Thread cache (tcache): per-thread cache of recently freed blocks for common sizes. Allows
malloc/freewithout any lock in the common case.
Problems:
- Multiple arenas fragment virtual memory
- Arena lock contention at high thread counts
- Poor performance for workloads with many concurrent allocations/frees
The numbers behind the design (glibc 2.26+ on 64-bit)
| Structure | Sizes served | Count / capacity | Behaviour |
|---|---|---|---|
| tcache | chunks up to 1032 bytes | 64 bins × 7 entries each, per thread | No locks, LIFO, no coalescing |
| fastbins | up to 128 bytes (default global_max_fast), max 160 | 10 bins, per arena | Singly linked LIFO, no coalescing until consolidation |
| unsorted bin | any | 1 per arena | Freed chunks land here first; sorted into small/large bins lazily on the next malloc |
| small bins | < 1024 bytes | 62 bins, exact 16-byte spacing | Doubly linked FIFO, coalesced |
| large bins | ≥ 1024 bytes | 63 bins, logarithmic ranges | Sorted by size, best-fit |
| top chunk | — | 1 per arena | Wilderness at the end of the heap; split when bins are empty |
| arenas | — | up to 8 × cores (64-bit) | Main arena uses brk; others use 64MB mmap'd heaps |
flowchart TD
M["malloc(n)"] --> T{"tcache bin for size has a chunk?"}
T -->|"yes"| R1["Return it (no lock, ~15ns)"]
T -->|"no"| BIG{"n above mmap threshold?"}
BIG -->|"yes"| MM["mmap a dedicated chunk"]
BIG -->|"no"| L["Lock this thread's arena"]
L --> FB{"fastbin or small bin exact fit?"}
FB -->|"yes"| R2["Return chunk, refill tcache from same bin"]
FB -->|"no"| UB["Process unsorted bin: return exact fit, sort others into bins"]
UB --> LB{"large bin best fit?"}
LB -->|"yes"| R3["Split, remainder to unsorted bin"]
LB -->|"no"| TOP{"Top chunk big enough?"}
TOP -->|"yes"| R4["Split top chunk"]
TOP -->|"no"| GROW["Grow heap: brk (main arena) or new mmap heap"]
The arena explosion
Each secondary arena reserves a 64MB region of address space (HEAP_MAX_SIZE). A JVM or Ruby process with 200 threads can end up with 8 × 32 cores = 256 arenas: tens of GB of VSZ and, more importantly, hundreds of partially used heaps that each hold onto freed memory. This is the famous "64MB blocks in pmap" signature:
pmap -x <pid> | awk '$2 >= 65000 && $2 <= 65536' | wc -l # count ~64MB arena heaps
The standard mitigation is to cap arenas:
export MALLOC_ARENA_MAX=2 # Heroku's documented recommendation for Ruby apps
# or, glibc 2.26+:
export GLIBC_TUNABLES=glibc.malloc.arena_max=2:glibc.malloc.trim_threshold=131072
This trades some lock contention for much less fragmentation. Heroku documents MALLOC_ARENA_MAX=2 for Ruby, and Hadoop/YARN's docs have long recommended MALLOC_ARENA_MAX=4 for JVM containers for exactly this reason.
Giving memory back: malloc_trim
#include <malloc.h>
malloc_trim(0); // return free memory to the OS: shrinks the brk top AND
// madvise(MADV_DONTNEED)s free pages inside every arena (glibc 2.8+)
malloc_stats(); // per-arena summary to stderr
malloc_info(0, stdout); // XML: per-arena, per-bin detail
struct mallinfo2 mi = mallinfo2(); // glibc 2.33+: 64-bit-safe counters (mallinfo() overflows past 2GB)
Calling malloc_trim(0) after a known spike (end of a batch, after cache eviction) can drop RSS dramatically on glibc. It walks all arenas under their locks, so don't call it on a hot path.
$ ./app # with malloc_stats() at exit
Arena 0:
system bytes = 135168000
in use bytes = 12582912
Arena 1:
system bytes = 67108864
in use bytes = 1048576
Total (incl. mmap):
system bytes = 202276864 ← RSS-ish: what glibc holds from the OS
in use bytes = 13631488 ← what the app is actually using: 7% utilisation
max mmap regions = 12
tcmalloc (Thread-Caching Malloc)
Google's allocator used in Chrome, production services. Designed for high-concurrency workloads.
Architecture:
Thread-local cache: per-thread pool of small objects (< 256KB)
↕ (refill/return in batches)
Central cache: per-size-class lists, one lock per size class
↕
Page heap: manages spans (runs of pages) from the OS
Key advantages:
- Thread-local cache makes most small allocations/frees lock-free
- Batch transfers between thread cache and central cache reduce lock frequency
- Low fragmentation — tight size classes minimize internal fragmentation
Performance: 2–8x faster than ptmalloc2 for highly concurrent workloads.
(Treat that range as "in allocation-heavy multi-threaded microbenchmarks". Modern glibc with tcache has closed much of the single-threaded gap; real application gains are usually single-digit to low-double-digit percent in CPU, and the bigger win is often memory.)
Two tcmallocs
There are two codebases with this name, and their behaviour differs:
| gperftools tcmalloc | google/tcmalloc (2020+) | |
|---|---|---|
| Front-end cache | Per-thread | Per-CPU (using Linux restartable sequences, rseq) |
| Huge pages | Oblivious | Hugepage-aware ("Temeraire") back end |
| Profiling | HEAPPROFILE, pprof | Built-in sampled profiling exposed to Google's profilers |
| Build | autotools, packaged in distros (libgoogle-perftools) | Bazel/CMake; less often packaged |
| Used by | Many open-source projects via distro packages | Google's fleet, MongoDB 8.0+ |
Per-CPU caches matter at high thread counts: with per-thread caches, 2,000 threads × a few MB of cache each is gigabytes of idle cached memory; with per-CPU caches, the number of caches is bounded by core count, and each one stays hot. rseq lets a thread operate on "its" CPU's cache without locks, aborting and retrying if it gets preempted or migrated mid-operation.
The hugepage-aware back end packs allocations so that whole 2MB regions are either densely used or entirely free (and can be released), which makes THP pay off instead of bloating RSS. Google's OSDI 2021 paper "Beyond malloc efficiency to fleet efficiency: a hugepage-aware memory allocator" reported fleet-wide throughput gains of several percent and reduced memory use — at Google's scale, a large number of machines' worth of savings.
jemalloc
Originally written by Jason Evans for FreeBSD (it became FreeBSD's libc allocator in 2005–2006), then developed for years at Facebook/Meta, where it backs most C++ services; Firefox uses a long-diverged fork (mozjemalloc), and Redis bundles it as its default allocator on Linux. Designed for low fragmentation and memory profiling.
Key features:
- Size classes: 232 size classes with careful spacing to minimize internal fragmentation
- Arenas: multiple arenas (default: 4× CPU count). Thread bound to an arena.
- Thread-local caching: per-thread cache, similar to tcmalloc
- Extent management: tracks memory at the "extent" (multi-page) level for better fragmentation control
- Built-in profiling: heap profiling with
MALLOC_CONF=prof:true— samples allocations and produces flame graphs
The size classes are spaced so that there are four classes per doubling (e.g. 64, 80, 96, 112, 128, 160, 192, 224, 256, …): worst-case internal fragmentation is bounded at about 20%, versus up to 50% for pure power-of-two classes. Small objects (up to 14KB with 4KB pages) live in slabs — runs of pages holding objects of one size class, with an allocation bitmap — and larger ones get their own extents.
Purging: how jemalloc gives memory back
This is the most important thing to understand about jemalloc in production. Free pages go through two stages before returning to the OS, each with a time-based decay:
stateDiagram-v2
[*] --> Active: allocated
Active --> Dirty: freed (still resident)
Dirty --> Muzzy: after dirty_decay_ms (MADV_FREE)
Muzzy --> Retained: after muzzy_decay_ms (MADV_DONTNEED)
Dirty --> Active: reused (no page fault)
Muzzy --> Active: reused
Retained --> Active: reused (new page faults)
- dirty pages are free but still resident — cheap to reuse.
- muzzy pages have been
MADV_FREEd — the kernel may reclaim them under pressure; RSS doesn't drop until it does. - Defaults:
dirty_decay_ms:10000,muzzy_decay_ms:0(since 5.2 on Linux) — free memory drifts back to the OS over about 10 seconds. background_thread:truedoes the purging on a background thread rather than inline inmalloc/freecalls, which smooths latency.
# Common production MALLOC_CONF
export MALLOC_CONF="background_thread:true,dirty_decay_ms:5000,narenas:8,metadata_thp:auto"
# Print statistics at exit
MALLOC_CONF=stats_print:true ./app 2>&1 | grep -E 'Allocated|active|resident|mapped|retained'
# Allocated: 1503912488, active: 1576431616, metadata: 32418112, resident: 1652400128,
# mapped: 1789136896, retained: 412090368
Read those as: allocated (bytes the app holds) ≤ active (in pages with live objects) ≤ resident (RSS attributable to jemalloc) ≤ mapped. resident / allocated is your fragmentation-plus-cache overhead — here about 1.10, which is very healthy.
In code, the same numbers are available via mallctl:
#include <jemalloc/jemalloc.h>
uint64_t epoch = 1; size_t sz = sizeof(epoch);
mallctl("epoch", &epoch, &sz, &epoch, sz); // refresh cached stats
size_t allocated, resident; sz = sizeof(size_t);
mallctl("stats.allocated", &allocated, &sz, NULL, 0);
mallctl("stats.resident", &resident, &sz, NULL, 0);
mallctl("arena.4096.purge", NULL, NULL, NULL, 0); // 4096 = MALLCTL_ARENAS_ALL: purge now
Why Redis uses jemalloc: Redis's memory usage must be predictable and reportable. jemalloc's fragmentation behavior is more consistent than ptmalloc2, and its reporting is more accurate.
It also makes Redis's activedefrag possible (above), and INFO memory reports allocator_allocated, allocator_active, allocator_resident straight from jemalloc's stats.
Project status
In 2025 jemalloc's original author archived the upstream GitHub repository and published a retrospective ("jemalloc Postmortem"). The allocator remains in wide use and large users maintain their own branches, but check the current maintenance status before choosing it for a new project; tcmalloc and mimalloc are actively developed alternatives.
mimalloc
Microsoft Research's allocator (Daan Leijen et al., 2019). Small (~10k lines), very fast, and with an unusual design centred on free-list sharding:
- Memory is divided into pages (64KB for small objects) each holding one size class, grouped into segments.
- Each page has three free lists instead of one per size class: a free list for allocation, a local-free list that the owning thread frees into, and a thread-free list that other threads free into atomically. Allocation pops from the free list with no atomics; when it's empty, the other two are swapped in in bulk.
- Because free lists are per page and short, allocations from the same page are close together in memory — good locality — and a page whose objects are all freed can be returned to the OS as a unit.
- A secure mode adds guard pages, encrypted free-list pointers and randomised allocation order to blunt heap exploits, for ~10% cost.
mimalloc is used inside Microsoft services, is the default in some language runtimes (Koka; CPython 3.13's free-threaded "no-GIL" build uses it internally), and is popular as a drop-in in Rust (mimalloc crate) and C# native code. On many allocation-heavy benchmarks it is as fast or faster than jemalloc and tcmalloc with similar or lower memory use; on some workloads with lots of cross-thread frees it can retain more memory.
// Rust: switch the global allocator in two lines
use mimalloc::MiMalloc;
#[global_allocator]
static GLOBAL: MiMalloc = MiMalloc;
// or tikv_jemallocator::Jemalloc, or tcmalloc via the `tcmalloc` crate
Choosing an Allocator
| Allocator | Best for |
|---|---|
| ptmalloc2 (glibc) | General purpose, single-threaded or low-contention |
| tcmalloc | High-concurrency servers, latency-sensitive (Google stack) |
| jemalloc | Servers where memory fragmentation matters (databases, caches) |
| mimalloc | General-purpose alternative, very fast for small allocations |
A more detailed comparison:
| glibc ptmalloc2 | jemalloc | tcmalloc (google) | mimalloc | |
|---|---|---|---|---|
| Fast path | per-thread tcache (7/bin) | per-thread tcache | per-CPU cache (rseq) | per-thread page free lists |
| Contention at 64+ threads | Arena locks; tune arena_max | Low | Very low | Low |
| Returning memory to OS | Poor (top-of-heap trim, malloc_trim) | Good, time-decay purging | Good, hugepage-aware | Good, per-page |
| Fragmentation in long-running servers | Often high | Low | Low | Low |
| Huge page friendliness | Poor | OK (metadata_thp, tuned decay) | Excellent | OK |
| Heap profiling | None built in (use external tools) | Excellent (prof:true, jeprof) | Excellent (sampled) | Basic stats |
Drop-in via LD_PRELOAD | — | Yes | Yes (gperftools easiest) | Yes |
| Typical RSS change switching from glibc | — | −10% to −50% on fragmenting workloads | similar | similar |
When to switch
Switch from glibc when you see any of:
- RSS climbing over days while the application's own accounting of live objects is flat
perfshows significant time inmalloc/free/_int_malloc/__lll_lock_waitunder concurrency- Many ~64MB anonymous mappings in
pmap(arena explosion) - You need heap profiling in production
Don't bother when the program is short-lived, single-threaded, or allocates little — and be careful in programs that mix allocators (memory malloced by one and freed by another is a crash), use dlopened plugins with static allocators, or rely on glibc-specific functions like malloc_trim.
# Try it with zero code changes
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2 ./server
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libtcmalloc_minimal.so.4 ./server
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libmimalloc.so ./server
# Docker: install the package and set ENV LD_PRELOAD=... in the Dockerfile
# Permanently: link with -ljemalloc (or -ltcmalloc), which takes precedence over libc's malloc
Production stories
- Ruby on Rails memory bloat. Ruby's many threads (Puma, Sidekiq) and glibc's arenas produced RSS growth that looked like leaks. Mike Perham (Sidekiq) and others documented in 2018 that running with jemalloc or
MALLOC_ARENA_MAX=2cut memory use by roughly half on typical Sidekiq workloads; Heroku madeMALLOC_ARENA_MAX=2part of its Ruby guidance, and jemalloc-enabled Ruby builds became common. - Rust dropped jemalloc as the default. Until Rust 1.32 (2019), binaries bundled jemalloc. It was removed in favour of the system allocator to cut binary size and platform issues; teams that need it opt back in via
#[global_allocator](as TiKV and many others do). - Redis and jemalloc. Redis switched its default to jemalloc on Linux in 2.4 (2011) after fragmentation issues with glibc, and later built active defragmentation on jemalloc's internals.
- MongoDB and tcmalloc. MongoDB has used tcmalloc for years; 8.0 moved to the newer per-CPU, hugepage-aware tcmalloc and changed its THP recommendation from "disable" to "enable" as a result.
Arena, Region and Pool Allocators
General-purpose allocators pay for generality: headers, size lookups, thread safety, and per-object free. When you know the lifetime pattern of your allocations, you can do dramatically better with a purpose-built allocator layered on top.
Bump / arena (region) allocators
Allocate by incrementing a pointer; free everything at once by resetting it. No per-object headers, no per-object free, O(1) and ~1–2 ns per allocation.
typedef struct { char *base, *ptr, *end; } Arena;
void *arena_alloc(Arena *a, size_t n) {
n = (n + 15) & ~(size_t)15; // 16-byte alignment
if (a->ptr + n > a->end) return NULL; // or chain a new block
void *p = a->ptr; a->ptr += n; return p;
}
void arena_reset(Arena *a) { a->ptr = a->base; } // "free" every object at once
// Rust: bumpalo — everything allocated in `bump` is freed when it's dropped
let bump = bumpalo::Bump::new();
for tok in tokens {
let node: &mut Node = bump.alloc(Node::from(tok)); // pointer bump, no malloc call
tree.push(node);
}
handle(&tree);
// `bump` dropped at end of scope: one deallocation for the whole request
Perfect for per-request or per-frame data: parsers and compilers (AST nodes), HTTP request handling, game frame allocations, query execution in databases (PostgreSQL's memory contexts, Apache APR pools, and nginx's ngx_pool_t are all region allocators — nginx frees a request's whole pool when the request ends, which is a big reason it doesn't leak). The risk: anything that must outlive the region needs to be copied out, and one long-lived reference keeps the whole region alive.
Go experimented with this as GOEXPERIMENT=arenas (Go 1.20); the proposal is on hold because it is hard to make memory-safe in a GC'd language — a pointer into a freed arena must fault rather than silently read reused memory.
Pool (free-list / object pool) allocators
Fixed-size objects recycled through a free list — the user-space version of the slab idea. Good for objects with expensive initialisation or very high churn: connection objects, buffers, message structs.
// Go: sync.Pool caches reusable objects per P and lets the GC clear them
var bufPool = sync.Pool{
New: func() any { return new(bytes.Buffer) },
}
func render(w io.Writer, v any) {
buf := bufPool.Get().(*bytes.Buffer)
buf.Reset()
defer bufPool.Put(buf)
_ = json.NewEncoder(buf).Encode(v)
w.Write(buf.Bytes())
}
Pools can leak and lie
A pool that never shrinks is a leak with good intentions: one burst of 10,000 concurrent requests leaves 10,000 buffers parked forever. Pools also hide allocations from profilers and defeat tools like ASan (use-after-return-to-pool isn't detected). sync.Pool avoids the first problem by letting the GC drop pooled items over two GC cycles; hand-rolled pools need a cap. And pooling large buffers of varied size can make fragmentation worse — bucket them by size class.
| Pattern | Alloc cost | Free cost | Best for | Watch out for |
|---|---|---|---|---|
| General malloc | 10–50 ns | 10–50 ns | Anything | Fragmentation, contention |
| Bump/arena | 1–2 ns | ~0 (bulk reset) | Same-lifetime batches | Lifetime bugs, retention |
| Fixed-size pool | ~5 ns | ~5 ns | Hot fixed-size objects | Unbounded growth |
| Stack allocation | ~0 | ~0 | Small, scoped data | Stack size limits |
Garbage Collected Languages
GC languages (Java, Go, Python) manage memory automatically, but still rely on an underlying allocator and have their own allocation strategies.
Go's allocator: Inspired by tcmalloc. Size classes from 8B to 32KB. Objects ≤ 32KB go to per-P (processor) caches; larger objects go directly to the heap. GC uses tri-color mark-and-sweep with write barriers.
JVM G1GC: Heap divided into equal-sized regions. Young regions collect frequently (minor GC). Old regions collect when full (major GC). Aims for predictable pause times.
Python: Reference counting for immediate deallocation + cycle detector for reference cycles. CPython has a small object allocator (pymalloc) for objects ≤ 512 bytes organized in arenas/pools/blocks.
Go and the OS
Go's allocator layers are mcache (per P, no locks) → mcentral (per size class, ~68 classes) → mheap (spans of 8KB pages, obtained from the OS in 64MB heap arenas). Objects that don't escape the function are put on the stack by the compiler and never touch the allocator at all (go build -gcflags=-m shows escape decisions).
The GC decides when to collect, and a separate scavenger decides when to give free pages back to the OS:
| Knob | Effect |
|---|---|
GOGC=100 (default) | Next GC when the heap grows 100% over live data after the last GC — so heap ≈ 2× live |
GOMEMLIMIT (Go 1.19+) | Soft total memory limit; GC runs more often as the limit approaches. Set ~90% of container limit |
GOGC=off + GOMEMLIMIT | Collect only when nearing the limit — good for batch jobs with a known budget |
| Scavenger | Returns pages with MADV_DONTNEED in the background, targeting retained memory close to the heap goal |
var m runtime.MemStats
runtime.ReadMemStats(&m)
fmt.Printf("HeapAlloc=%d HeapInuse=%d HeapIdle=%d HeapReleased=%d Sys=%d NumGC=%d\n",
m.HeapAlloc, m.HeapInuse, m.HeapIdle, m.HeapReleased, m.Sys, m.NumGC)
// RSS ≈ HeapInuse + (HeapIdle − HeapReleased) + stacks + runtime metadata
Why Go RSS sometimes didn't drop (MADV_FREE)
Go 1.12–1.15 returned memory with MADV_FREE, which lets the kernel reclaim lazily — so RSS stayed high until there was memory pressure, and container dashboards showed "leaks" that weren't. Go 1.16 switched back to MADV_DONTNEED by default (RSS drops immediately). The same effect exists in jemalloc's muzzy pages and in any allocator using MADV_FREE: RSS is not the same as "memory that can't be reclaimed".
JVM: heap vs RSS
The single most common JVM memory surprise: -Xmx4g and a container OOM-killed at 6GB. The heap is only one of many consumers:
| Region | Typical size | Controlled by |
|---|---|---|
| Java heap | -Xmx | -Xmx, -XX:MaxRAMPercentage |
| Metaspace (class metadata) | 50–500 MB | -XX:MaxMetaspaceSize |
| Code cache (JIT output) | 50–250 MB | -XX:ReservedCodeCacheSize |
| Thread stacks | 1 MB reserved per thread (-Xss), less resident | thread count |
| GC data structures | 2–10% of heap (G1 remembered sets, card tables) | collector choice |
| Direct / NIO buffers | can be GBs (Netty, Kafka clients) | -XX:MaxDirectMemorySize |
| Native allocations via glibc (JNI, zlib, SSL) | varies | MALLOC_ARENA_MAX, alternative allocator |
| Arena fragmentation in glibc | can be hundreds of MB | MALLOC_ARENA_MAX=2..4, jemalloc |
java -XX:NativeMemoryTracking=summary -jar app.jar &
jcmd <pid> VM.native_memory summary
# Total: reserved=7834MB, committed=5921MB
# - Java Heap (reserved=4096MB, committed=4096MB)
# - Class (reserved=1101MB, committed=161MB)
# - Thread (reserved=412MB, committed=412MB) ← 400 threads
# - Code (reserved=250MB, committed=98MB)
# - GC (reserved=210MB, committed=210MB)
# - Internal / Other (direct buffers) ...
If NMT's committed total is well below RSS, the gap is usually native allocations outside the JVM's view — glibc arenas from JNI libraries — which is where MALLOC_ARENA_MAX or LD_PRELOADing jemalloc earns its keep (and jemalloc's profiler can then tell you which native call sites allocate).
Python
pymalloc handles objects ≤ 512 bytes in pools (one size class each) inside arenas (256KB before Python 3.10, 1MB since), and returns an arena to the OS only when every object in it is freed — so a few surviving objects can pin whole arenas, the same pinned-page problem as glibc. Larger objects go to the system malloc. PYTHONMALLOC=malloc bypasses pymalloc (useful with valgrind or to try jemalloc); tracemalloc is the built-in allocation tracker:
import tracemalloc
tracemalloc.start(25)
# ... run workload ...
snap = tracemalloc.take_snapshot()
for stat in snap.statistics("lineno")[:10]:
print(stat) # file:line size=48.2 MiB count=402113 average=126 B
Observability
# Linux: per-process memory breakdown
cat /proc/<pid>/status # VmRSS (resident), VmSize (virtual)
pmap -x <pid> # per-mapping breakdown
# Heap profiling with jemalloc
MALLOC_CONF=prof:true,prof_final:true ./app
jeprof --pdf app heap.prof > heap.pdf
# Heap profiling with tcmalloc (gperftools)
LD_PRELOAD=/usr/lib/libtcmalloc.so HEAPPROFILE=/tmp/heap ./app
pprof --pdf ./app /tmp/heap.0001.heap > heap.pdf
# Valgrind: memory error detection (slow but thorough)
valgrind --leak-check=full ./app
# AddressSanitizer: fast memory error detection (~2x slowdown)
clang -fsanitize=address -g ./app.c -o app && ./app
(Corrections to the original snippet: heap profiling in gperftools lives in libtcmalloc.so — libprofiler.so is the CPU profiler; jemalloc must be built with --enable-prof for prof:true to work, and it writes files named jeprof.<pid>.<seq>.f.heap at exit, so the jeprof command takes that filename.)
Leak, fragmentation, or cache? Decide first
"Memory keeps growing" has three different causes needing three different tools. Work out which one you have before reaching for valgrind:
flowchart TD
A["RSS grows over time"] --> B{"Does the app's own live-object count grow too? (heap profile, GC live bytes)"}
B -->|"yes"| C{"Are the objects reachable?"}
C -->|"yes, intentionally"| CACHE["Unbounded cache or queue: add bounds or eviction"]
C -->|"no (C/C++) or unexpectedly (GC)"| LEAK["Leak: heaptrack, jemalloc prof, LSan, pprof diff"]
B -->|"no, flat"| D{"Allocator resident much greater than allocated?"}
D -->|"yes"| FRAG["Fragmentation or retention: arena_max, malloc_trim, switch allocator, decay settings"]
D -->|"no"| E["Outside the heap: thread stacks, mmaps, page cache, tmpfs, kernel memory - check smaps and memory.stat"]
The toolbox
| Tool | Finds | Overhead | Use in production? |
|---|---|---|---|
| Valgrind memcheck | Leaks, invalid reads/writes, use of uninitialised memory | 20–50× slower | No — tests only |
Valgrind massif + ms_print | Heap usage over time and peak contributors | ~20× | No |
| AddressSanitizer / LeakSanitizer | Overflows, use-after-free, leaks at exit | ~2× CPU, 2–3× memory | Canary or staging |
| heaptrack | Every allocation with stack traces; leaks, peak, temporary allocations | ~2–5× | Short sessions on a canary |
jemalloc profiling (prof:true,lg_prof_sample:19) | Sampled live allocations by stack (every ~512KB) | ~1–5% | Yes |
| tcmalloc / gperftools heap profiler | Sampled allocations | Low | Yes |
bcc memleak (eBPF) | Outstanding allocations by stack, for any process, no restart | Moderate (traces every malloc) | Briefly |
Go pprof heap | inuse_space / alloc_space by stack | ~0 (sampled) | Yes |
| JVM NMT, heap dumps, JFR | Native categories, object graphs, allocation sampling | Low–moderate | Yes (JFR) |
malloc_stats / malloc_info | glibc arena utilisation | ~0 | Yes |
# heaptrack: record and analyse
heaptrack ./app # writes heaptrack.app.<pid>.zst
heaptrack --analyze heaptrack.app.12345.zst | head -60 # or heaptrack_gui
# "PEAK MEMORY CONSUMERS", "MOST TEMPORARY ALLOCATIONS", "MEMORY LEAKED" sections
# jemalloc: profile a running service, dumping every 1GB of allocation growth
export MALLOC_CONF="prof:true,prof_active:true,lg_prof_sample:19,lg_prof_interval:30,prof_prefix:/tmp/jeprof"
./server &
# later: compare two dumps to see what grew
jeprof --text --base=/tmp/jeprof.1234.3.i3.heap ./server /tmp/jeprof.1234.9.i9.heap | head -20
# eBPF: who is holding unfreed memory right now? (bcc-tools)
/usr/share/bcc/tools/memleak -p <pid> --top 10 30
# [12:04:31] Top 10 stacks with outstanding allocations:
# 104857600 bytes in 25600 allocations from stack
# malloc+0x... lru_insert+0x52 [server] handle_set+0x1a0 [server] ...
# Go
curl -s localhost:6060/debug/pprof/heap > h1.pb.gz; sleep 600
curl -s localhost:6060/debug/pprof/heap > h2.pb.gz
go tool pprof -top -diff_base=h1.pb.gz h2.pb.gz
Diff, don't stare
A single heap profile shows what's big; a leak shows up as what grows. Take two profiles some time apart under steady load and diff them (--base, -diff_base). The top of the diff is almost always the leak.
For deeper eBPF-based allocation tracing, see eBPF.
Real Numbers
| Quantity | Value |
|---|---|
Fast-path malloc/free (tcache, jemalloc, tcmalloc, mimalloc) | ~10–20 ns |
| Contended glibc arena lock path | 100 ns – several µs |
brk/mmap syscall | ~0.2–1 µs + page faults on first touch |
| Minor page fault on fresh anonymous memory | ~0.5–2 µs per 4KB |
| glibc minimum chunk / overhead per allocation (64-bit) | 32 bytes / 8 bytes |
glibc mmap threshold | 128 KiB (dynamic up to 32 MiB) |
| glibc max arenas (64-bit) | 8 × cores, 64 MiB reserved each |
| glibc tcache | 64 bins × 7 chunks, ≤ 1032 bytes |
| jemalloc size classes / spacing | ~232 / 4 per doubling (≤ ~20% internal waste) |
jemalloc default dirty_decay_ms | 10 s |
| Go size classes / max small object | ~68 / 32 KiB |
| pymalloc small object limit | 512 bytes |
| Linux buddy max order | 10 (4 MiB) |
| Typical RSS reduction switching glibc → jemalloc on fragmenting servers | 10–50% |
Interview Questions
"What is heap fragmentation? How do allocators deal with it?" External fragmentation: free memory is split into chunks too small to satisfy large allocations. Internal fragmentation: allocated blocks are larger than requested (due to size rounding). Allocators combat fragmentation with size classes (allocations rounded to one of ~50 sizes, reducing internal fragmentation), coalescing (merging adjacent free blocks to reduce external fragmentation), and thread-local caches (reducing the spread of allocations across the heap).
"Why would you use jemalloc instead of the default glibc malloc?" jemalloc has better fragmentation characteristics for long-running servers — the glibc allocator tends to accumulate fragmentation over time, causing RSS to grow unboundedly even when the application's logical heap size is stable. jemalloc's more careful size class design and arena management keep fragmentation bounded. Also: built-in heap profiling.
"What happens when you call malloc() at a system level?"
For small allocations: check thread-local cache (no lock). If empty, refill from a size-class free list (lock per size class). If empty, allocate pages from the OS via mmap(MAP_ANONYMOUS). For large allocations (>128KB typically): call mmap directly and return the mapping. free() returns the block to the thread cache or to the size-class free list.
(In glibc specifically, the main arena grows with brk and secondary arenas with mmap; and nothing physical is allocated until the pages are first touched and page-fault.)
"Why doesn't RSS go down after I free memory?"
free returns memory to the allocator, not the OS. glibc can only shrink its main heap from the top and otherwise keeps freed memory for reuse; other allocators return pages on a timer (jemalloc decay, Go scavenger) or lazily via MADV_FREE, which leaves RSS high until the kernel needs the memory. Fragmentation — a few live objects per page — can prevent return entirely.
"How does the Linux kernel allocate memory for its own objects?"
Pages come from the buddy allocator (power-of-two blocks of pages, buddies found by XOR, coalesced on free). Small, frequent objects come from SLUB caches — per-type slabs of fixed-size objects with a per-CPU fast path — and kmalloc uses generic size-class caches.
"A JVM with -Xmx4g uses 6GB of RSS. Where's the rest?"
Metaspace, code cache, thread stacks, GC structures, direct buffers, and native allocations through glibc (plus arena fragmentation). Measure with Native Memory Tracking; if NMT's total is below RSS, look at glibc arenas (MALLOC_ARENA_MAX) or switch the native allocator.
"When would you write your own allocator?" When allocation lifetimes are structured: per-request or per-frame data (arena — allocate by bumping a pointer, free everything at once), or many identical hot objects (pool). Not to beat jemalloc at general-purpose allocation.
Related Pages
- Virtual Memory — pages, faults,
mmap, THP, RSS/PSS, container limits - System Calls & Kernel Internals —
brk,mmap,madviseas syscalls and their cost - Linux Memory Management — the operator's view
- Redis Internals — jemalloc, fragmentation ratio, active defrag
- Go Concurrency — goroutine stacks and the Go runtime
- Rust for Backend —
#[global_allocator], ownership-based memory management