V8 & Node.js Internals
Most performance advice for JavaScript is a list of rules: "don't use delete", "keep object shapes consistent", "don't block the event loop". The rules are right, but without the machinery behind them they are cargo cult. This page is the "how it works inside" layer: what V8 does between reading your source file and running machine code, how it decides to throw that machine code away, how memory is reclaimed, and how Node.js glues V8 to the operating system through libuv.
For tuning advice built on top of this, see the optimization pages: V8 Optimization, Node.js Event Loop, Memory Management, and Worker Threads. For measuring what the engine is doing, see Node.js Profiling. For the general theory of interpreters and JITs, see Compilers & Interpreters.
The Big Picture
A Node.js process is three cooperating systems:
flowchart TB
subgraph JS["Your JavaScript"]
APP["app.js, node_modules"]
end
subgraph NODE["Node.js core"]
LIB["lib/*.js<br/>(fs, net, http, stream)"]
BIND["C++ bindings<br/>(internalBinding)"]
end
subgraph V8["V8 engine"]
PARSER["Parser"]
TIERS["Ignition / Sparkplug /<br/>Maglev / TurboFan"]
HEAP["Heap + Orinoco GC"]
end
subgraph UV["libuv"]
LOOP["Event loop"]
POOL["Thread pool"]
POLL["epoll / kqueue / IOCP"]
end
OS["Operating system"]
APP --> LIB
LIB --> BIND
APP --> PARSER
PARSER --> TIERS
TIERS --> HEAP
BIND --> LOOP
BIND --> POOL
LOOP --> POLL
POLL --> OS
POOL --> OS
| Component | Responsibility | Written in | Threads |
|---|---|---|---|
| V8 | Parse, compile, execute JS; manage the JS heap | C++ | Main thread + background compile and GC helper threads |
| Node core (JS) | Public APIs (fs, net, http) | JavaScript | Main thread |
| Node core (C++) | Bindings from JS objects to libuv/OpenSSL/zlib | C++ | Main thread, plus pool threads for work items |
| libuv | Event loop, async I/O, timers, thread pool | C | Main thread + 4 pool threads by default |
The key insight: your JavaScript runs on exactly one thread, but the process has many. V8 compiles and collects garbage in the background; libuv does file I/O and crypto in its pool; the kernel handles socket readiness. Almost every performance problem in Node is either "the one JS thread is doing too much" or "the JS thread is waiting on a resource that is saturated."
Part 1: The V8 Execution Pipeline
From Source to Bytecode
flowchart LR
SRC["Source text<br/>(UTF-8/16)"] --> SCAN["Scanner<br/>(tokens)"]
SCAN --> PRE["Preparser<br/>(lazy: syntax check,<br/>scope info only)"]
SCAN --> FULL["Full parser<br/>(AST)"]
PRE -.->|"function first called"| FULL
FULL --> BCG["Bytecode generator"]
BCG --> BC["Bytecode +<br/>constant pool"]
BC --> IGN["Ignition interpreter"]
Lazy parsing. Parsing every function eagerly would waste time and memory on code that never runs (most of a typical node_modules tree). V8 therefore preparses inner functions: it checks syntax and records which variables they capture from enclosing scopes, but does not build an AST or generate bytecode. When the function is first called, V8 fully parses it and generates bytecode.
The cost of lazy parsing is that a function called immediately gets parsed twice. V8 uses heuristics to parse eagerly when it can tell a function will run right away. The classic one is the parenthesized function expression (the "PIFE" heuristic):
// Preparsed, then fully parsed on first call
const a = function () { /* ... */ };
// The leading paren hints "this will be invoked immediately" -> parsed eagerly
const b = (function () { /* ... */ })();
Bundlers such as Rollup and esbuild used to wrap modules in parentheses specifically to trigger this. Modern V8 also uses the code cache: after compilation, bytecode can be serialized and reused on the next run, skipping parsing entirely. Node uses this for its own core modules (baked into the startup snapshot), and since Node 22 you can enable it for your own modules with module.enableCompileCache() or the NODE_COMPILE_CACHE environment variable, which typically cuts cold-start time for large CLIs noticeably.
Ignition: A Register Machine With an Accumulator
Ignition bytecode is compact and designed for a register machine with a special accumulator register that most operations implicitly read and write. You can see it:
node --print-bytecode --print-bytecode-filter=add -e '
function add(a, b) { return a + b; }
for (let i = 0; i < 10; i++) add(i, 1);
'
Parameter count 3
Register count 0
Frame size 0
0 : Ldar a1 ; load register a1 (b) into accumulator
2 : Add a0, [0] ; accumulator = a0 + accumulator, feedback slot 0
5 : Return ; return accumulator
Two details matter for performance:
[0]is a feedback slot. Every operation whose behavior depends on runtime types (property loads, calls, arithmetic, comparisons) has a slot index into the function's feedback vector. This is how the interpreter records what it saw.- Bytecode is ~25-50% the size of baseline machine code. Ignition was introduced (2016-2017) largely to cut memory on low-end phones, not for speed. The interpreter itself is generated by V8's own compiler backend, so it is fast for an interpreter, but still roughly an order of magnitude slower than optimized code on hot arithmetic.
Feedback Vectors
A feedback vector is a per-closure-family array of slots, one per type-dependent operation:
flowchart LR
FN["JSFunction 'getX'"] --> FC["FeedbackCell"]
FC --> FV["FeedbackVector"]
FV --> S0["Slot 0: LoadIC<br/>state: monomorphic<br/>map: Point -> offset 12"]
FV --> S1["Slot 1: BinaryOp<br/>seen: SignedSmall"]
FV --> S2["Slot 2: Call<br/>target: Math.sqrt"]
FV --> BUD["Interrupt budget<br/>(tier-up counter)"]
Feedback vectors are allocated lazily — a function that runs once or twice never gets one, which saves significant memory on large codebases. Once allocated, each slot accumulates what the code has actually done:
| Slot kind | Records | Used by optimizer to |
|---|---|---|
| Load/Store property | Hidden class(es) seen and the field offset or handler | Emit a map check + direct memory load |
| Keyed load/store | Elements kind(s) seen | Emit a bounds check + typed array access |
| Binary/compare op | Type lattice: SignedSmall, Number, String, BigInt, Any | Emit integer add with overflow check, float add, etc. |
| Call | Target function(s) | Inline the callee |
| Literal | Allocation site | Pre-size objects/arrays, choose elements kind |
The interrupt budget is the tier-up mechanism. Every function call and every loop back-edge decrements it, weighted by bytecode size. When it runs out, V8 checks whether the function deserves a higher tier. This is why "hot" is not a fixed call count: a big function with a hot loop can tier up after a single call, a tiny function after many.
The Four Tiers
flowchart LR
IGN["Ignition<br/>interpreter"] -->|"budget exhausted,<br/>some feedback"| SP["Sparkplug<br/>baseline compiler"]
SP -->|"warm, stable<br/>feedback"| MAG["Maglev<br/>mid-tier optimizer"]
MAG -->|"hot, long-running"| TF["TurboFan<br/>top-tier optimizer"]
MAG -.->|"deopt"| IGN
TF -.->|"deopt"| IGN
| Tier | Since | Input | Output quality | Compile speed | Speculates? |
|---|---|---|---|---|---|
| Ignition | V8 5.9 (2017) | AST | Bytecode (interpreted) | Fastest | No — only records feedback |
| Sparkplug | V8 9.1 (2021) | Bytecode | Machine code, one template per bytecode | Very fast (no IR) | No |
| Maglev | Chrome 117 (2023) | Bytecode + feedback | SSA machine code with basic optimizations | ~10x faster than TurboFan | Yes |
| TurboFan | 2017 as top tier | Bytecode + feedback | Heavily optimized machine code (inlining, escape analysis, bounds-check elimination) | Slow; runs mostly on a background thread | Yes, aggressively |
Sparkplug is the clever one. It walks bytecode and emits a fixed machine-code snippet for each instruction, calling the same builtins Ignition would. It keeps the exact interpreter frame layout, so it never needs to deoptimize and can be swapped in or out at any point. It removes interpreter dispatch overhead (decode, jump table) without trying to be smart.
Maglev fills the gap between "fast to compile, slow to run" and "slow to compile, fast to run." It builds an SSA graph directly from bytecode and feedback, does representation selection (unboxed int32/float64), inlining of small functions, and simple checks elimination, then does straightforward register allocation. Most web and server code spends most of its optimized time in Maglev.
TurboFan is the heavyweight: speculative inlining, escape analysis (to eliminate allocations of objects that never leave the function), loop peeling, redundancy elimination, and typed lowering. Historically it used a "sea of nodes" IR; V8 has been migrating its backend and then its JavaScript pipeline to Turboshaft, a CFG-based IR that compiles faster with similar output. You do not need to care which IR is active, but you will see both names in V8 flags and traces.
You can watch the tiering happen:
node --trace-opt --trace-deopt app.js 2>&1 | grep -E 'marking|completed|bailout|deoptimizing'
[completed compiling 0x2a1c... <JSFunction parseRow> (target MAGLEV) - took 0.412 ms]
[marking 0x2a1c... <JSFunction parseRow> for optimization to TURBOFAN_JS, ConcurrencyMode::kConcurrent, reason: hot and stable]
[completed compiling 0x2a1c... <JSFunction parseRow> (target TURBOFAN_JS) - took 4.873 ms]
Note the ~10x difference in compile time — that is why the middle tier exists.
On-Stack Replacement
Tier-up normally takes effect on the next call. A function that is called once but runs a loop for ten seconds would never benefit. On-stack replacement (OSR) solves this: when the loop back-edge exhausts the budget, V8 compiles an optimized version with an entry point in the middle of the loop, then transfers the live interpreter frame into the optimized frame and jumps there. In traces you will see OSR in the reason. OSR code is often slightly worse than normally-entered optimized code, which is one reason microbenchmarks that run one giant loop at top level are misleading.
Speculative Optimization
The fundamental trick of a JS JIT: compile for what has happened, guard against what might. Consider:
function lengthSquared(p) {
return p.x * p.x + p.y * p.y;
}
In Ignition, p.x is a generic property lookup and * handles every type combination. If the feedback says "p always had map M1, x and y are always small integers," TurboFan generates something like:
; p in rax
test al, 1 ; is p a heap object (not a Smi)?
jz DEOPT_0
cmp [rax-1], M1 ; CheckMaps: is p's hidden class M1?
jne DEOPT_1
mov ecx, [rax+0x0b] ; load p.x at fixed offset (int32 field)
imul ecx, ecx ; x * x
jo DEOPT_2 ; overflow -> result is not a Smi
mov edx, [rax+0x0f] ; load p.y
imul edx, edx
jo DEOPT_3
add ecx, edx
jo DEOPT_4
; ... box and return
Every DEOPT_n label is a deoptimization exit: an escape hatch that fires if the assumption that justified the fast path turns out false.
Deoptimization
sequenceDiagram
participant OptCode as Optimized code
participant Deopt as Deoptimizer
participant FV as Feedback vector
participant Ign as Ignition
OptCode->>OptCode: CheckMaps fails (p has map M2)
OptCode->>Deopt: jump to deopt exit #1
Deopt->>Deopt: read deopt data:<br/>which bytecode offset,<br/>where each live value lives
Deopt->>Deopt: materialize escaped-away objects,<br/>rebox unboxed numbers
Deopt->>Ign: build interpreter frame(s),<br/>resume at bytecode offset
Deopt->>FV: feedback now includes M2
Note over Ign,FV: Function may re-optimize later<br/>with polymorphic feedback
Reconstructing interpreter frames is non-trivial: optimized code may have inlined three functions, kept values in registers, and eliminated allocations entirely. The optimizer records deoptimization data at every exit describing how to rebuild each frame.
| Deopt type | Trigger | Example |
|---|---|---|
| Eager | A guard inside optimized code fails right now | Map check fails, Smi overflow, array index out of bounds |
| Lazy | Code was invalidated by a change elsewhere; deopt happens when control returns to it | Someone modified Array.prototype, or a field's representation generalized from Smi to Double |
| Soft | Optimizer hit an operation with no feedback and inserted an unconditional deopt | A branch that never ran during warm-up |
A single deopt is cheap (microseconds). The pathology is a deopt loop: optimize, deopt, re-optimize, deopt. V8 counts deopts per function and eventually gives up optimizing it, leaving it in a lower tier permanently. Common reasons in real traces:
[bailout (kind: deopt-eager, reason: not a Smi): ...]
[bailout (kind: deopt-eager, reason: out of bounds): ...]
[bailout (kind: deopt-lazy, reason: (unknown)): ...]
[bailout (kind: deopt-eager, reason: Insufficient type feedback for call): ...]
Reading deopt reasons
"wrong map" means a shape changed — look at object construction. "not a Smi" / "lost precision" means integers became floats or overflowed 31/32 bits. "out of bounds" means reading past an array's length (often a loop with <=). "Insufficient type feedback" during warm-up is normal; repeated ones in steady state are not. Fix patterns are in V8 Optimization.
Part 2: Objects, Shapes, and Inline Caches
Hidden Classes (Maps)
JavaScript objects look like hash maps, but treating them as hash maps would make every property access a hash lookup. V8 instead gives each object a pointer to a Map (called a hidden class or shape in other engines) that describes its layout. Objects built the same way share a Map.
flowchart LR
M0["Map M0<br/>(empty)"] -->|"add 'x'"| M1["Map M1<br/>x @ offset 0"]
M1 -->|"add 'y'"| M2["Map M2<br/>x @ 0, y @ 1"]
M1 -->|"add 'z'"| M3["Map M3<br/>x @ 0, z @ 1"]
M0 -->|"add 'y'"| M4["Map M4<br/>y @ 0"]
M4 -->|"add 'x'"| M5["Map M5<br/>y @ 0, x @ 1"]
Maps form a transition tree. Adding a property follows (or creates) a transition edge. {x, y} and {y, x} end up on different Maps (M2 vs M5) even though they have the same keys, because property order is part of the layout.
What a Map stores:
| Field | Purpose |
|---|---|
| Descriptor array | Property names, attributes, field index, field representation |
| Field representation | Smi, Double, HeapObject, or Tagged per field |
| Prototype | So prototype-chain lookups can be cached by Map |
| Instance size / in-object property count | How many properties live inside the object vs. a separate backing store |
| Elements kind | How indexed properties are stored (see below) |
| Transitions | Edges to child Maps |
| Stability / deprecation bits | Whether optimized code may depend on this Map not changing |
In-object vs. out-of-object properties. V8 pre-allocates some slots directly inside the object. Properties beyond that go into a separate properties backing store (one extra pointer hop). For constructor functions and classes, V8 uses slack tracking: the first several instances are allocated generously, and once V8 has seen how many properties instances actually get, it shrinks the instance size for future allocations. This is why initializing all fields in the constructor matters — late-added fields often land out-of-object.
Field representation generalization. If field x has always held Smis, the Map records Smi and optimized code can load it as an integer without checks. Store a float into it and the field generalizes to Double (stored unboxed or as a HeapNumber); store an object and it becomes Tagged. Generalization deprecates the old Map and lazily deoptimizes code that depended on it. Generalization only ever goes one way, so a single stray assignment permanently weakens the field for every object of that shape.
Dictionary mode. When an object gets too many properties added dynamically, or you delete a property that is not the last one added, V8 converts it to dictionary mode: a real hash table with its own Map that says "slow." Property access works but no inline cache can use a fixed offset. Objects used as hash maps (cache[key] = value with arbitrary keys) are in dictionary mode by design — which is fine, and also why Map is usually a better choice for that job.
You can inspect all of this with natives syntax:
// node --allow-natives-syntax shapes.js
class Point { constructor(x, y) { this.x = x; this.y = y; } }
const a = new Point(1, 2);
const b = new Point(3, 4);
console.log(%HaveSameMap(a, b)); // true
const c = new Point(1, 2);
c.z = 3; // transition to a child Map
console.log(%HaveSameMap(a, c)); // false
const d = new Point(1.5, 2); // x generalizes Smi -> Double
console.log(%HaveSameMap(a, d)); // true again: a's Map was deprecated and migrated
%DebugPrint(a); // prints map, fields, representations
Inline Caches
An inline cache (IC) is the feedback slot plus a fast-path handler for one operation site. Each IC moves through states:
stateDiagram-v2
[*] --> Uninitialized
Uninitialized --> Monomorphic: first execution, 1 map
Monomorphic --> Polymorphic: 2nd distinct map
Polymorphic --> Polymorphic: up to 4 maps
Polymorphic --> Megamorphic: 5th distinct map
Monomorphic --> Megamorphic: repeated map deprecation
Megamorphic --> [*]
| State | Maps seen | Lookup cost | Optimizer output |
|---|---|---|---|
| Monomorphic | 1 | One map compare + load at fixed offset | CheckMaps + direct load; can inline |
| Polymorphic | 2-4 | Linear compare against each map | Branch per map; still inlinable |
| Megamorphic | 5+ | Global stub cache (hashed by map and name), falls back to full lookup on miss | Generic call to a load builtin; no inlining |
A megamorphic site is not catastrophically slow in isolation — the global stub cache is a reasonably fast hash. The real cost is what it blocks: the optimizer cannot inline, cannot eliminate checks, and cannot do escape analysis through it. In tight loops this is often 5-20x versus monomorphic, and the effect compounds through every function that calls it.
// One site, many shapes -> megamorphic
function getId(entity) { return entity.id; }
getId({ id: 1 });
getId({ id: 2, name: 'a' });
getId({ name: 'b', id: 3 }); // different order -> different map
getId(new User(4));
getId(new Order(5)); // 5 maps: megamorphic
// Monomorphic alternative: normalize shape at the boundary
class EntityRef { constructor(id, kind) { this.id = id; this.kind = kind; } }
The classic real-world source of megamorphism is a generic utility (get(obj, path), an ORM's row.toJSON(), a logging serializer) that sees every shape in the program at one property-access site. The fix is either shape normalization upstream or accepting that the generic utility lives off the hot path.
Elements Kinds
Indexed properties (arr[0]) are stored separately from named properties, in an elements backing store. The Map records the elements kind, which lets optimized code skip type checks per element.
flowchart LR
PS["PACKED_SMI_ELEMENTS"] --> PD["PACKED_DOUBLE_ELEMENTS"]
PD --> PE["PACKED_ELEMENTS"]
PS --> HS["HOLEY_SMI_ELEMENTS"]
PD --> HD["HOLEY_DOUBLE_ELEMENTS"]
PE --> HE["HOLEY_ELEMENTS"]
HS --> HD
HD --> HE
HE --> DICT["DICTIONARY_ELEMENTS<br/>(sparse)"]
Transitions only go down and right — toward more general. There is no way back for that array.
| Operation | Resulting kind | Why it matters |
|---|---|---|
[1, 2, 3] | PACKED_SMI | Elements are raw tagged Smis; fastest |
arr.push(1.5) | PACKED_DOUBLE | Stored as unboxed float64 array; still fast |
arr.push('a') | PACKED_ELEMENTS | Each element is a tagged pointer; type check per read |
new Array(3) | HOLEY_SMI | Holes must check the prototype chain on read |
arr[1000] = 1 on short array | DICTIONARY | Hash table; very slow iteration |
arr.push(-0) or NaN into Smi array | PACKED_DOUBLE | -0 and NaN are not Smis |
Holes are worse than they look: reading a hole means "not present here, check Array.prototype and Object.prototype," so the optimizer must either insert a prototype-chain check or depend on the prototypes being pristine (a protector — a global flag that, if invalidated by someone patching Array.prototype, lazily deopts every dependent function in the process).
Typed arrays sidestep all of this
Float64Array, Int32Array, and friends have a fixed element type and no holes. They are backed by an ArrayBuffer outside the normal elements store, so numeric hot paths (parsers, image processing, simulations) avoid elements-kind transitions entirely. They also cross to native code without copying, which matters for addons (see Part 4).
Value Representation: Smis, HeapNumbers, and Pointer Compression
Every JS value in V8 is a tagged word. The low bit distinguishes a Smi (small integer, stored inline) from a pointer to a heap object. Floats, large integers, strings, and objects are all heap objects.
| Build | Tagged word | Smi range | Notes |
|---|---|---|---|
| 64-bit, no pointer compression (Node default) | 64 bits | 32-bit signed | Pointers are full 64-bit |
| 64-bit with pointer compression (Chrome default) | 32 bits | 31-bit signed | Heap confined to a 4 GB "cage"; ~40% less heap memory |
Node.js builds without pointer compression by default, so its heap is not limited to 4 GB per isolate — but each pointer costs 8 bytes rather than 4. If you compare memory use between Chrome and Node for the same code, this is a significant part of the difference.
A number that is not a Smi — 0.5, 2 ** 40, -0 — is boxed as a HeapNumber unless the optimizer can keep it unboxed in a register or a Double-represented field. Arithmetic-heavy code that keeps "accidentally" producing floats allocates on every operation in unoptimized tiers.
Part 3: Memory and the Orinoco Garbage Collector
Heap Layout
flowchart TB
subgraph HEAP["V8 heap (per isolate)"]
subgraph YG["Young generation"]
NURS["Semi-space A<br/>(allocation)"]
SURV["Semi-space B<br/>(copy target)"]
end
subgraph OG["Old generation"]
OLD["Old space<br/>(survived objects)"]
CODE["Code space<br/>(JIT output)"]
LO["Large object space<br/>(objects above ~ page size)"]
TR["Trusted / read-only spaces"]
end
end
EXT["External memory<br/>(ArrayBuffer backing stores,<br/>Buffer, native addon memory)"]
HEAP -.->|"tracked by size,<br/>not by GC tracing"| EXT
V8's collector is built on the generational hypothesis: most objects die young. Request-scoped objects (parsed JSON, intermediate arrays, closures for a promise chain) usually die before the request finishes. A few objects (caches, module state, connection pools) live forever.
| Space | Collected by | Typical size | Algorithm |
|---|---|---|---|
| Young generation | Scavenger (minor GC) | Semi-spaces of a few MB to 16 MB+ each (--max-semi-space-size) | Parallel copying (Cheney-style) |
| Old space | Mark-compact (major GC) | Up to --max-old-space-size | Concurrent mark, parallel compact, concurrent sweep |
| Large object space | Major GC | Objects too big for a normal page | Never moved; marked and freed |
| Code space | Major GC | JIT output | Mark-compact, with flushing of old bytecode |
The Scavenger (Minor GC)
sequenceDiagram
participant M as Main thread
participant H as Helper threads
participant FS as From-space
participant TS as To-space / Old space
Note over M: Allocation fails: young gen full
M->>M: Pause JS
M->>H: Distribute roots<br/>(stack, globals, remembered set)
par Parallel scavenge
H->>FS: Find live object
H->>TS: Copy to to-space<br/>(or promote to old space if it survived once)
H->>FS: Leave forwarding pointer
end
M->>M: Swap semi-spaces#59;<br/>from-space is now empty
Note over M: Resume JS (pause: ~0.5-5 ms)
Allocation in the young generation is a pointer bump — just as fast as a stack allocation. Collection cost is proportional to the live objects, not the dead ones; garbage is free. That is why allocating many short-lived objects is usually fine, while allocating objects that survive one scavenge and then die (promoted, then garbage in old space) is expensive: they pay for a copy and then burden the major GC.
Write barriers and the remembered set. To collect the young generation without scanning the whole old generation, V8 must know which old objects point to young ones. Every pointer store in JIT code includes a write barrier: a few instructions that, if an old object is made to point at a young object, record the slot in a remembered set. Those slots become extra roots for the scavenger. The barrier is also used by concurrent marking (below). It is part of why storing pointers is more expensive than storing numbers.
Mark-Compact (Major GC)
Major GC has three phases, and Orinoco's contribution (2016-2019) was making each one concurrent, parallel, or incremental so that the main thread pauses as little as possible.
flowchart LR
subgraph MARK["1. Marking"]
IM["Incremental steps<br/>on main thread"]
CM["Concurrent marking<br/>on helper threads"]
FIN["Final atomic pause:<br/>re-scan roots, finish worklist"]
end
subgraph SWEEP["2. Sweeping"]
CS["Concurrent sweeping:<br/>build free lists"]
end
subgraph COMPACT["3. Compaction"]
PC["Parallel evacuation<br/>of fragmented pages"]
UP["Update pointers"]
end
IM --> FIN
CM --> FIN
FIN --> CS
FIN --> PC
PC --> UP
Marking uses the tri-color abstraction: white (not yet seen), grey (seen, children not yet scanned), black (fully scanned). Helper threads drain grey worklists while JS keeps running. The danger is that JS creates a pointer from a black object to a white object behind the marker's back; the marking write barrier catches that store and greys the target. A short final pause rescans the stack and finishes leftovers.
Sweeping walks pages and builds free lists from dead memory. It runs concurrently; the allocator can lazily sweep a page on demand if it needs space before the sweeper reaches it.
Compaction moves objects out of highly fragmented pages so those pages can be released. It is done in parallel during a pause, and only for pages where fragmentation makes it worth it.
| Technique | What runs on the main thread | Effect |
|---|---|---|
| Stop-the-world | Everything | Simple, long pauses (hundreds of ms on large heaps) |
| Parallel | Everything, but helpers speed it up | Shorter pauses, still a pause |
| Incremental | Small slices interleaved with JS | Many tiny pauses; total work slightly higher |
| Concurrent | Almost nothing; helpers work while JS runs | Lowest pause, needs barriers, competes for CPU cores |
Containers and helper threads
Concurrent GC assumes spare cores. In a container limited to one CPU (cpus: 1 or a CFS quota of 100ms/100ms), "concurrent" marking competes with your JS thread for the same core, and parallel scavenges cannot actually run in parallel. GC pauses that are invisible on a laptop become visible in production. Give Node services at least 2 vCPUs or measure GC time explicitly with --trace-gc and perf_hooks.
Observing GC
node --trace-gc server.js
[48213:0x5d1f...] 2104 ms: Scavenge 38.2 (52.1) -> 31.0 (53.1) MB, pooled: 0 MB, 2.11 / 0.00 ms (average mu = 0.991, current mu = 0.987) allocation failure;
[48213:0x5d1f...] 9871 ms: Mark-Compact (reduce) 212.4 (240.6) -> 120.7 (178.3) MB, 41.20 / 0.00 ms (+ 85.3 ms in 112 steps since start of marking, biggest step 3.1 ms, walltime since start of marking 902 ms) (average mu = 0.934, current mu = 0.912) finalize incremental marking via task; GC in old space requested
Read it as: kind before (committed) -> after (committed) MB, main-thread pause ms. The mu (mutator utilization) is the fraction of time JS got to run; below ~0.9 sustained means GC is a real cost. The parenthetical "+ 85.3 ms in 112 steps" is incremental work spread across the marking window.
Programmatic tracking for dashboards:
import { PerformanceObserver, constants } from 'node:perf_hooks';
const kinds = {
[constants.NODE_PERFORMANCE_GC_MINOR]: 'scavenge',
[constants.NODE_PERFORMANCE_GC_MAJOR]: 'mark-compact',
[constants.NODE_PERFORMANCE_GC_INCREMENTAL]: 'incremental',
[constants.NODE_PERFORMANCE_GC_WEAKCB]: 'weak-callbacks',
};
const obs = new PerformanceObserver((list) => {
for (const entry of list.getEntries()) {
const kind = kinds[entry.detail.kind] ?? 'other';
gcDurationHistogram.observe({ kind }, entry.duration); // e.g. prom-client Histogram
}
});
obs.observe({ entryTypes: ['gc'] });
Heap Limits and External Memory
| Flag | Default (64-bit, recent Node) | When to change |
|---|---|---|
--max-old-space-size=<MB> | Derived from system memory; in containers recent Node versions respect cgroup limits | Set to ~75% of container memory so the process GCs harder before the OOM killer acts |
--max-semi-space-size=<MB> | ~16 MB | Raise (32-64 MB) for allocation-heavy services to cut scavenge frequency and premature promotion; costs RSS |
--expose-gc | off | Tests and benchmarks only |
Buffer and ArrayBuffer contents live outside the V8 heap as external memory. V8 tracks their size (so a lot of external memory pressures GC to run), but --max-old-space-size does not bound it. A process with a 512 MB heap limit can have 2 GB RSS from buffers. When RSS and heapUsed diverge, check process.memoryUsage().external and arrayBuffers first. Leak-hunting technique is covered in Memory Management.
Part 4: libuv and the Event Loop
Architecture
libuv gives Node a single cross-platform abstraction over very different OS I/O models:
| Platform | Socket readiness | File I/O | Notes |
|---|---|---|---|
| Linux | epoll | Thread pool (io_uring for some ops in newer libuv, gated in Node) | Most production Node |
| macOS / BSD | kqueue | Thread pool | |
| Windows | IOCP (completion-based) | IOCP / thread pool | Completion vs. readiness model hidden by libuv |
The crucial split: network I/O is truly asynchronous at the OS level, so a single thread can wait on 100,000 sockets with one epoll_wait call. Regular file I/O on Linux is not (readiness APIs report regular files as always ready, and blocking reads can still stall on disk), so libuv runs it on a thread pool and reports completion back to the loop.
flowchart TB
subgraph MAIN["Main thread"]
JS["V8: your JS callbacks"]
LOOP["uv_run() loop"]
end
subgraph POOL["libuv thread pool (default 4)"]
T1["worker 1"]
T2["worker 2"]
T3["worker 3"]
T4["worker 4"]
end
Q["Work queue (FIFO)"]
K["Kernel: epoll/kqueue/IOCP"]
AS["uv_async wakeup<br/>(eventfd / pipe)"]
JS -->|"fs.readFile, crypto.pbkdf2,<br/>dns.lookup, zlib"| Q
Q --> T1
Q --> T2
Q --> T3
Q --> T4
T1 -->|"done"| AS
AS --> LOOP
JS -->|"socket read/write"| K
K -->|"readiness events"| LOOP
LOOP --> JS
Event Loop Phases
flowchart TB
START(["loop iteration"]) --> TIMERS["timers<br/>setTimeout / setInterval<br/>callbacks whose time has come"]
TIMERS --> PENDING["pending callbacks<br/>I/O callbacks deferred<br/>from the previous iteration"]
PENDING --> IDLE["idle, prepare<br/>(internal)"]
IDLE --> POLL["poll<br/>wait for I/O (epoll_wait)<br/>run I/O callbacks"]
POLL --> CHECK["check<br/>setImmediate callbacks"]
CHECK --> CLOSE["close callbacks<br/>socket.on('close')"]
CLOSE --> ALIVE{"active handles<br/>or requests?"}
ALIVE -->|"yes"| START
ALIVE -->|"no"| EXIT(["process exits"])
| Phase | Runs | Blocks? |
|---|---|---|
| timers | Expired setTimeout/setInterval callbacks, ordered by expiry | No |
| pending callbacks | Some system callbacks (e.g. certain TCP errors) deferred from the last poll | No |
| idle / prepare | libuv internals; Node uses prepare to start the event-loop-delay timer | No |
| poll | Computes timeout from the nearest timer, blocks in epoll_wait, runs I/O callbacks | Yes — this is where an idle process sleeps |
| check | setImmediate callbacks | No |
| close | 'close' events of handles closed with uv_close | No |
Microtasks are not a phase. After every callback that Node invokes from the loop (since Node 11), it drains the process.nextTick queue completely, then the promise microtask queue completely, repeating until both are empty. That has two consequences:
- A promise chain that keeps resolving synchronously (or recursive
nextTick) starves the loop indefinitely — no I/O, no timers. - A
setImmediatescheduled from an I/O callback always runs before asetTimeout(fn, 0)scheduled from the same callback, because check comes before the next timers phase. From the main module the order is nondeterministic. Detailed ordering rules and examples are in Node.js Event Loop.
What Actually Uses the Thread Pool
This is the single most useful table on this page for production debugging:
| API | Mechanism | Uses thread pool? |
|---|---|---|
net, http, https sockets, tls | Readiness via epoll/kqueue/IOCP | No |
fs.* async and fs.promises (read, write, stat, open, readdir...) | Blocking syscalls on pool threads | Yes |
fs.watch | inotify / FSEvents / ReadDirectoryChangesW | No |
dns.lookup() (default for http.get, net.connect by hostname) | getaddrinfo(3) — blocking libc call | Yes |
dns.resolve*(), dns.Resolver | c-ares over sockets | No |
crypto.pbkdf2, scrypt, randomBytes (async), randomFill, generateKeyPair, async hkdf | Pool work | Yes |
crypto.createHash, createCipheriv, sign | Synchronous on main thread | No (but blocks JS!) |
zlib async (gzip, deflate, brotliCompress) | Pool work | Yes |
child_process | fork/spawn + pipes | No (pipes are polled) |
| Timers | Min-heap in the loop | No |
Node-API napi_create_async_work | Pool work | Yes |
The default pool size is 4, set via the UV_THREADPOOL_SIZE environment variable (maximum 1024). It is sized once, the first time anything uses the pool, so it must be set before startup — setting process.env.UV_THREADPOOL_SIZE late in your code is unreliable.
sequenceDiagram
participant R as 100 requests
participant Q as Pool queue
participant P as 4 pool threads
participant F as fs.readFile (2 ms)
R->>Q: 50 x crypto.pbkdf2 (100 ms each)
R->>Q: 50 x fs.readFile
Q->>P: threads busy with pbkdf2 for 1.25 s
Note over F: readFile waits behind<br/>all 50 hash jobs:<br/>2 ms becomes ~1250 ms
P->>F: finally scheduled
War Story (illustrative)
The DNS lookup that slowed down file uploads
A service accepted uploads, wrote them to local disk, and posted metadata to an internal API by hostname. A resolver change made getaddrinfo slow (~2 s per lookup on cache miss). Every outbound request occupied a pool thread for 2 seconds inside dns.lookup, and with only four threads, the fs.write calls for uploads queued behind them. Upload latency jumped from 40 ms to 6 seconds while CPU sat at 5%. Nothing in the JS profile was hot; the event loop delay histogram was flat. The clue was the thread pool: four threads, all inside getaddrinfo.
The fix was three lines: a caching lookup function passed to the HTTP agent, UV_THREADPOOL_SIZE=16, and a dashboard panel for pool queue depth. The lesson: "async" in Node does not mean "free" — it means "runs somewhere else that can also saturate."
Monitoring the Loop
import { monitorEventLoopDelay, performance } from 'node:perf_hooks';
const delay = monitorEventLoopDelay({ resolution: 10 });
delay.enable();
let lastElu = performance.eventLoopUtilization();
setInterval(() => {
const elu = performance.eventLoopUtilization(lastElu); // delta since last sample
lastElu = performance.eventLoopUtilization();
metrics.gauge('nodejs.eventloop.delay.p99_ms', delay.percentile(99) / 1e6);
metrics.gauge('nodejs.eventloop.delay.max_ms', delay.max / 1e6);
metrics.gauge('nodejs.eventloop.utilization', elu.utilization); // 0..1
delay.reset();
}, 10_000).unref();
| Signal | Healthy | Investigate |
|---|---|---|
| Event loop delay p99 | < 20 ms | > 100 ms: something is blocking the JS thread (sync crypto, JSON of huge payloads, regex backtracking, GC) |
| Event loop utilization | < 0.7 | > 0.9 sustained: the JS thread is the bottleneck; scale out or move work to Worker Threads |
| High latency, low ELU, low CPU | — | Waiting on something external: thread pool saturation, DB pool, downstream service |
Part 5: Crossing the JS/Native Boundary
How a Core API Reaches C++
Calling fs.readFile is several layers deep:
sequenceDiagram
participant U as Your JS
participant L as lib/fs.js
participant B as internalBinding('fs')<br/>(C++ in src/node_file.cc)
participant UV as libuv
participant T as Pool thread
participant OS as Kernel
U->>L: fs.readFile(path, cb)
L->>L: validate args,<br/>create FSReqCallback
L->>B: binding.open(path, flags, mode, req)
B->>UV: uv_fs_open(loop, req, ..., AfterOpen)
UV->>T: enqueue work
T->>OS: open(2) — blocks this thread
OS-->>T: fd
T-->>UV: post completion
UV-->>B: AfterOpen (on main thread)
B-->>L: req.oncomplete(err, fd)
L->>B: binding.read(...) (and so on)
L-->>U: cb(null, data)
Points that explain real behavior:
internalBindingis how Node's own JS reaches its C++ layer. The older publicprocess.binding()is deprecated; do not depend on it.fs.readFileis not one operation. It isopen,fstat, one or morereads, andclose— each a separate pool trip. For a large file, that is many round trips through the queue, which is whyfs.readFileof a big file under pool contention is so much slower thanfs.createReadStreamwith a sensiblehighWaterMark, or a singlefs.promises.readFileon a quiet system.- Callbacks arrive on the main thread in the poll phase. The pool threads never touch V8; they cannot, because a V8 isolate is single-threaded.
Node-API and C++ Addons
There are three generations of addon interfaces:
| Interface | ABI stable across Node versions? | Status |
|---|---|---|
| Raw V8/libuv APIs | No — recompile for every major Node | Avoid for new code |
| NAN (Native Abstractions for Node) | No — source-compatible macros only | Legacy |
Node-API (napi_*, C) + node-addon-api (C++ wrapper) | Yes | Use this |
Node-API is a C ABI that hides V8 entirely: you work with opaque napi_value handles, and a binary compiled against Node-API version N runs on every later Node release. That stability is why prebuilt binaries (prebuildify, node-gyp-build) work.
A synchronous function with node-addon-api:
// src/sum.cc
#include <napi.h>
Napi::Value SumFloat64(const Napi::CallbackInfo& info) {
Napi::Env env = info.Env();
if (info.Length() < 1 || !info[0].IsTypedArray() ||
info[0].As<Napi::TypedArray>().TypedArrayType() != napi_float64_array) {
Napi::TypeError::New(env, "expected Float64Array").ThrowAsJavaScriptException();
return env.Null();
}
auto arr = info[0].As<Napi::Float64Array>();
// Zero-copy: points directly into the ArrayBuffer's backing store
const double* data = arr.Data();
size_t n = arr.ElementLength();
double total = 0.0;
for (size_t i = 0; i < n; ++i) total += data[i];
return Napi::Number::New(env, total);
}
Napi::Object Init(Napi::Env env, Napi::Object exports) {
exports.Set("sumFloat64", Napi::Function::New(env, SumFloat64));
return exports;
}
NODE_API_MODULE(sum, Init)
An asynchronous one that runs on the libuv pool. The key rule: Execute() runs on a pool thread and must not touch any JS value; copy inputs out first and build JS results in OnOK() back on the main thread.
// src/hash_async.cc
#include <napi.h>
#include <string>
#include <vector>
class HashWorker : public Napi::AsyncWorker {
public:
HashWorker(Napi::Function& cb, std::vector<uint8_t> input, uint32_t rounds)
: Napi::AsyncWorker(cb), input_(std::move(input)), rounds_(rounds) {}
// Pool thread: no Napi::Value, no V8, no exceptions to JS
void Execute() override {
if (rounds_ == 0) { SetError("rounds must be > 0"); return; } // rejects in OnError
uint64_t h = 1469598103934665603ULL; // FNV-1a offset basis
for (uint32_t r = 0; r < rounds_; ++r) {
for (uint8_t b : input_) { h ^= b; h *= 1099511628211ULL; }
}
result_ = h;
}
// Main thread: safe to create JS values
void OnOK() override {
Napi::Env env = Env();
Callback().Call({ env.Null(), Napi::BigInt::New(env, result_) });
}
private:
std::vector<uint8_t> input_;
uint32_t rounds_;
uint64_t result_ = 0;
};
Napi::Value HashAsync(const Napi::CallbackInfo& info) {
auto buf = info[0].As<Napi::Buffer<uint8_t>>();
uint32_t rounds = info[1].As<Napi::Number>().Uint32Value();
Napi::Function cb = info[2].As<Napi::Function>();
// Copy: the Buffer could be mutated or collected while the worker runs
std::vector<uint8_t> copy(buf.Data(), buf.Data() + buf.Length());
(new HashWorker(cb, std::move(copy), rounds))->Queue(); // self-deleting
return info.Env().Undefined();
}
{
"targets": [{
"target_name": "native",
"sources": ["src/sum.cc", "src/hash_async.cc"],
"include_dirs": ["<!@(node -p \"require('node-addon-api').include\")"],
"defines": ["NAPI_VERSION=8", "NAPI_DISABLE_CPP_EXCEPTIONS"],
"cflags_cc": ["-O3"]
}]
}
The Cost of Crossing
Every call into native code pays for argument checking, handle creation, and conversion. Rough orders of magnitude on a modern x86 server:
| Call | Approximate overhead | Implication |
|---|---|---|
| JS -> JS (inlined) | ~0 ns | The optimizer erased the call |
| JS -> JS (not inlined) | 2-5 ns | |
| JS -> Node-API function | 20-100 ns | Fine for coarse calls, ruinous per element |
| Passing a string to native | O(length): UTF-16 -> UTF-8 copy | Pass Buffer/TypedArray for large data |
Passing a TypedArray | ~constant: pointer to backing store | Zero-copy |
| Async work round trip | 10-50 µs + pool queueing | Only worth it for work > ~100 µs |
V8 has a Fast API mechanism that lets the optimizer call certain C++ functions directly with unboxed arguments, bypassing most of this overhead; Node uses it internally for hot bindings (some Buffer, perf_hooks, and url paths). It is not part of the stable Node-API surface for addon authors.
When native is worth it
Native code wins when the work per call is large and the data crosses as typed arrays: image codecs, compression, crypto primitives, numeric kernels. It loses when called per item in a hot loop with strings or objects — the boundary cost exceeds the speedup, and you also lose inlining. Before writing C++, try a Worker Thread (for parallelism) or WebAssembly (for portable compute); see WebAssembly.
Threads, Isolates, and Workers
A V8 isolate is an independent heap plus its own compiler state; it may only be entered by one thread at a time. worker_threads creates a new isolate and a new libuv loop on a new OS thread — which is why workers do not share JS objects and communicate by structured clone or SharedArrayBuffer. All workers in a process share the one libuv thread pool, so moving pool-heavy work into workers does not add pool capacity.
| Primitive | New isolate? | New event loop? | Shares heap? | Shares thread pool? |
|---|---|---|---|---|
worker_threads.Worker | Yes | Yes | No (only SharedArrayBuffer) | Yes |
child_process.fork | Yes (new process) | Yes | No | No |
cluster worker | Yes (new process) | Yes | No | No |
| Async work in an addon | No | No | — | Yes (it is the pool) |
Putting It Together: A Request's Life
sequenceDiagram
participant K as Kernel
participant L as libuv poll
participant H as http parser (llhttp, C)
participant J as Your handler (JIT code)
participant G as GC
participant P as Thread pool
K->>L: socket readable (epoll)
L->>H: on read callback
H->>J: 'request' event -> handler
J->>J: JSON.parse body<br/>(young-gen allocations)
J->>P: crypto.scrypt(password)
Note over J: returns to loop#59;<br/>microtasks drain
G->>G: scavenge (1-2 ms pause)<br/>parse garbage freed for free
P-->>L: scrypt done
L->>J: callback -> respond
J->>K: socket write (may complete synchronously)
Each stage maps to a failure mode you can now name:
| Symptom | Layer | First thing to check |
|---|---|---|
| Throughput drops after a deploy, no CPU change | V8 tiers | --trace-deopt for deopt loops in hot functions |
CPU high, flame graph shows LoadIC_Megamorphic / KeyedLoadIC_Megamorphic | Inline caches | A generic accessor seeing many shapes |
| Periodic latency spikes every few seconds | GC | --trace-gc, major GC pause, mutator utilization |
RSS grows, heapUsed flat | External memory | process.memoryUsage().arrayBuffers, native addon leaks |
| Latency high, CPU low, loop delay low | Thread pool | Pool-bound APIs (dns.lookup, fs, crypto, zlib) and pool size |
| Latency high, loop delay high | Main thread | CPU profile for sync work |
Diagnostic Flag Reference
| Flag | Shows |
|---|---|
--print-bytecode --print-bytecode-filter=fn | Ignition bytecode for a function |
--trace-opt / --trace-deopt | Tier-up decisions and deopt reasons |
--log-ic | Writes IC state transitions to the V8 log (process with v8-tools / Indicium) |
--trace-maps | Map creation and transitions (very verbose) |
--allow-natives-syntax | Enables %DebugPrint, %HaveSameMap, %GetOptimizationStatus, %OptimizeFunctionOnNextCall |
--no-opt, --no-maglev, --no-sparkplug | Disable tiers to isolate a JIT-related difference |
--trace-gc, --trace-gc-verbose | Every GC with pause times |
--max-old-space-size, --max-semi-space-size | Heap sizing (MB) |
--prof + node --prof-process | Sampling profile including VM-internal ticks (GC, IC misses, compiler) |
--cpu-prof, --heap-prof | .cpuprofile / .heapprofile for DevTools |
Natives syntax is for investigation only
% functions are internal, change between V8 versions, and can crash the process with bad arguments. Use them in scratch scripts to understand behavior; never ship them, and never make performance decisions from a microbenchmark that forces optimization with %OptimizeFunctionOnNextCall without also checking real-traffic behavior.
Further Reading
- V8 Optimization — writing code that stays in the fast tiers
- Node.js Event Loop — exact phase ordering, starvation patterns, and loop tuning
- Memory Management — leak detection, heap snapshots, WeakRef and FinalizationRegistry
- Worker Threads — parallelism patterns and pooling
- Node.js Profiling — flame graphs,
--cpu-prof, clinic.js - Compilers & Interpreters — the general theory behind JIT tiers
- Bun Runtime — a JavaScriptCore-based alternative with a different internal design
- Memory Allocators and I/O & Storage — the OS layer beneath libuv
"Every rule about fast JavaScript is a sentence about an engine assumption. Learn the assumptions and you will not need the rules."