On this page

V8 & Node.js Internals

Most performance advice for JavaScript is a list of rules: "don't use delete", "keep object shapes consistent", "don't block the event loop". The rules are right, but without the machinery behind them they are cargo cult. This page is the "how it works inside" layer: what V8 does between reading your source file and running machine code, how it decides to throw that machine code away, how memory is reclaimed, and how Node.js glues V8 to the operating system through libuv.

For tuning advice built on top of this, see the optimization pages: V8 Optimization, Node.js Event Loop, Memory Management, and Worker Threads. For measuring what the engine is doing, see Node.js Profiling. For the general theory of interpreters and JITs, see Compilers & Interpreters.

The Big Picture

A Node.js process is three cooperating systems:

flowchart TB
    subgraph JS["Your JavaScript"]
        APP["app.js, node_modules"]
    end
    subgraph NODE["Node.js core"]
        LIB["lib/*.js<br/>(fs, net, http, stream)"]
        BIND["C++ bindings<br/>(internalBinding)"]
    end
    subgraph V8["V8 engine"]
        PARSER["Parser"]
        TIERS["Ignition / Sparkplug /<br/>Maglev / TurboFan"]
        HEAP["Heap + Orinoco GC"]
    end
    subgraph UV["libuv"]
        LOOP["Event loop"]
        POOL["Thread pool"]
        POLL["epoll / kqueue / IOCP"]
    end
    OS["Operating system"]

    APP --> LIB
    LIB --> BIND
    APP --> PARSER
    PARSER --> TIERS
    TIERS --> HEAP
    BIND --> LOOP
    BIND --> POOL
    LOOP --> POLL
    POLL --> OS
    POOL --> OS
ComponentResponsibilityWritten inThreads
V8Parse, compile, execute JS; manage the JS heapC++Main thread + background compile and GC helper threads
Node core (JS)Public APIs (fs, net, http)JavaScriptMain thread
Node core (C++)Bindings from JS objects to libuv/OpenSSL/zlibC++Main thread, plus pool threads for work items
libuvEvent loop, async I/O, timers, thread poolCMain thread + 4 pool threads by default

The key insight: your JavaScript runs on exactly one thread, but the process has many. V8 compiles and collects garbage in the background; libuv does file I/O and crypto in its pool; the kernel handles socket readiness. Almost every performance problem in Node is either "the one JS thread is doing too much" or "the JS thread is waiting on a resource that is saturated."

Part 1: The V8 Execution Pipeline

From Source to Bytecode

flowchart LR
    SRC["Source text<br/>(UTF-8/16)"] --> SCAN["Scanner<br/>(tokens)"]
    SCAN --> PRE["Preparser<br/>(lazy: syntax check,<br/>scope info only)"]
    SCAN --> FULL["Full parser<br/>(AST)"]
    PRE -.->|"function first called"| FULL
    FULL --> BCG["Bytecode generator"]
    BCG --> BC["Bytecode +<br/>constant pool"]
    BC --> IGN["Ignition interpreter"]

Lazy parsing. Parsing every function eagerly would waste time and memory on code that never runs (most of a typical node_modules tree). V8 therefore preparses inner functions: it checks syntax and records which variables they capture from enclosing scopes, but does not build an AST or generate bytecode. When the function is first called, V8 fully parses it and generates bytecode.

The cost of lazy parsing is that a function called immediately gets parsed twice. V8 uses heuristics to parse eagerly when it can tell a function will run right away. The classic one is the parenthesized function expression (the "PIFE" heuristic):

// Preparsed, then fully parsed on first call
const a = function () { /* ... */ };

// The leading paren hints "this will be invoked immediately" -> parsed eagerly
const b = (function () { /* ... */ })();

Bundlers such as Rollup and esbuild used to wrap modules in parentheses specifically to trigger this. Modern V8 also uses the code cache: after compilation, bytecode can be serialized and reused on the next run, skipping parsing entirely. Node uses this for its own core modules (baked into the startup snapshot), and since Node 22 you can enable it for your own modules with module.enableCompileCache() or the NODE_COMPILE_CACHE environment variable, which typically cuts cold-start time for large CLIs noticeably.

Ignition: A Register Machine With an Accumulator

Ignition bytecode is compact and designed for a register machine with a special accumulator register that most operations implicitly read and write. You can see it:

node --print-bytecode --print-bytecode-filter=add -e '
function add(a, b) { return a + b; }
for (let i = 0; i < 10; i++) add(i, 1);
'
Parameter count 3
Register count 0
Frame size 0
   0 : Ldar a1              ; load register a1 (b) into accumulator
   2 : Add a0, [0]          ; accumulator = a0 + accumulator, feedback slot 0
   5 : Return               ; return accumulator

Two details matter for performance:

  1. [0] is a feedback slot. Every operation whose behavior depends on runtime types (property loads, calls, arithmetic, comparisons) has a slot index into the function's feedback vector. This is how the interpreter records what it saw.
  2. Bytecode is ~25-50% the size of baseline machine code. Ignition was introduced (2016-2017) largely to cut memory on low-end phones, not for speed. The interpreter itself is generated by V8's own compiler backend, so it is fast for an interpreter, but still roughly an order of magnitude slower than optimized code on hot arithmetic.

Feedback Vectors

A feedback vector is a per-closure-family array of slots, one per type-dependent operation:

flowchart LR
    FN["JSFunction 'getX'"] --> FC["FeedbackCell"]
    FC --> FV["FeedbackVector"]
    FV --> S0["Slot 0: LoadIC<br/>state: monomorphic<br/>map: Point -> offset 12"]
    FV --> S1["Slot 1: BinaryOp<br/>seen: SignedSmall"]
    FV --> S2["Slot 2: Call<br/>target: Math.sqrt"]
    FV --> BUD["Interrupt budget<br/>(tier-up counter)"]

Feedback vectors are allocated lazily — a function that runs once or twice never gets one, which saves significant memory on large codebases. Once allocated, each slot accumulates what the code has actually done:

Slot kindRecordsUsed by optimizer to
Load/Store propertyHidden class(es) seen and the field offset or handlerEmit a map check + direct memory load
Keyed load/storeElements kind(s) seenEmit a bounds check + typed array access
Binary/compare opType lattice: SignedSmall, Number, String, BigInt, AnyEmit integer add with overflow check, float add, etc.
CallTarget function(s)Inline the callee
LiteralAllocation sitePre-size objects/arrays, choose elements kind

The interrupt budget is the tier-up mechanism. Every function call and every loop back-edge decrements it, weighted by bytecode size. When it runs out, V8 checks whether the function deserves a higher tier. This is why "hot" is not a fixed call count: a big function with a hot loop can tier up after a single call, a tiny function after many.

The Four Tiers

flowchart LR
    IGN["Ignition<br/>interpreter"] -->|"budget exhausted,<br/>some feedback"| SP["Sparkplug<br/>baseline compiler"]
    SP -->|"warm, stable<br/>feedback"| MAG["Maglev<br/>mid-tier optimizer"]
    MAG -->|"hot, long-running"| TF["TurboFan<br/>top-tier optimizer"]
    MAG -.->|"deopt"| IGN
    TF -.->|"deopt"| IGN
TierSinceInputOutput qualityCompile speedSpeculates?
IgnitionV8 5.9 (2017)ASTBytecode (interpreted)FastestNo — only records feedback
SparkplugV8 9.1 (2021)BytecodeMachine code, one template per bytecodeVery fast (no IR)No
MaglevChrome 117 (2023)Bytecode + feedbackSSA machine code with basic optimizations~10x faster than TurboFanYes
TurboFan2017 as top tierBytecode + feedbackHeavily optimized machine code (inlining, escape analysis, bounds-check elimination)Slow; runs mostly on a background threadYes, aggressively

Sparkplug is the clever one. It walks bytecode and emits a fixed machine-code snippet for each instruction, calling the same builtins Ignition would. It keeps the exact interpreter frame layout, so it never needs to deoptimize and can be swapped in or out at any point. It removes interpreter dispatch overhead (decode, jump table) without trying to be smart.

Maglev fills the gap between "fast to compile, slow to run" and "slow to compile, fast to run." It builds an SSA graph directly from bytecode and feedback, does representation selection (unboxed int32/float64), inlining of small functions, and simple checks elimination, then does straightforward register allocation. Most web and server code spends most of its optimized time in Maglev.

TurboFan is the heavyweight: speculative inlining, escape analysis (to eliminate allocations of objects that never leave the function), loop peeling, redundancy elimination, and typed lowering. Historically it used a "sea of nodes" IR; V8 has been migrating its backend and then its JavaScript pipeline to Turboshaft, a CFG-based IR that compiles faster with similar output. You do not need to care which IR is active, but you will see both names in V8 flags and traces.

You can watch the tiering happen:

node --trace-opt --trace-deopt app.js 2>&1 | grep -E 'marking|completed|bailout|deoptimizing'
[completed compiling 0x2a1c... <JSFunction parseRow> (target MAGLEV) - took 0.412 ms]
[marking 0x2a1c... <JSFunction parseRow> for optimization to TURBOFAN_JS, ConcurrencyMode::kConcurrent, reason: hot and stable]
[completed compiling 0x2a1c... <JSFunction parseRow> (target TURBOFAN_JS) - took 4.873 ms]

Note the ~10x difference in compile time — that is why the middle tier exists.

On-Stack Replacement

Tier-up normally takes effect on the next call. A function that is called once but runs a loop for ten seconds would never benefit. On-stack replacement (OSR) solves this: when the loop back-edge exhausts the budget, V8 compiles an optimized version with an entry point in the middle of the loop, then transfers the live interpreter frame into the optimized frame and jumps there. In traces you will see OSR in the reason. OSR code is often slightly worse than normally-entered optimized code, which is one reason microbenchmarks that run one giant loop at top level are misleading.

Speculative Optimization

The fundamental trick of a JS JIT: compile for what has happened, guard against what might. Consider:

function lengthSquared(p) {
  return p.x * p.x + p.y * p.y;
}

In Ignition, p.x is a generic property lookup and * handles every type combination. If the feedback says "p always had map M1, x and y are always small integers," TurboFan generates something like:

    ; p in rax
    test   al, 1                  ; is p a heap object (not a Smi)?
    jz     DEOPT_0
    cmp    [rax-1], M1            ; CheckMaps: is p's hidden class M1?
    jne    DEOPT_1
    mov    ecx, [rax+0x0b]        ; load p.x at fixed offset (int32 field)
    imul   ecx, ecx               ; x * x
    jo     DEOPT_2                ; overflow -> result is not a Smi
    mov    edx, [rax+0x0f]        ; load p.y
    imul   edx, edx
    jo     DEOPT_3
    add    ecx, edx
    jo     DEOPT_4
    ; ... box and return

Every DEOPT_n label is a deoptimization exit: an escape hatch that fires if the assumption that justified the fast path turns out false.

Deoptimization

sequenceDiagram
    participant OptCode as Optimized code
    participant Deopt as Deoptimizer
    participant FV as Feedback vector
    participant Ign as Ignition

    OptCode->>OptCode: CheckMaps fails (p has map M2)
    OptCode->>Deopt: jump to deopt exit #1
    Deopt->>Deopt: read deopt data:<br/>which bytecode offset,<br/>where each live value lives
    Deopt->>Deopt: materialize escaped-away objects,<br/>rebox unboxed numbers
    Deopt->>Ign: build interpreter frame(s),<br/>resume at bytecode offset
    Deopt->>FV: feedback now includes M2
    Note over Ign,FV: Function may re-optimize later<br/>with polymorphic feedback

Reconstructing interpreter frames is non-trivial: optimized code may have inlined three functions, kept values in registers, and eliminated allocations entirely. The optimizer records deoptimization data at every exit describing how to rebuild each frame.

Deopt typeTriggerExample
EagerA guard inside optimized code fails right nowMap check fails, Smi overflow, array index out of bounds
LazyCode was invalidated by a change elsewhere; deopt happens when control returns to itSomeone modified Array.prototype, or a field's representation generalized from Smi to Double
SoftOptimizer hit an operation with no feedback and inserted an unconditional deoptA branch that never ran during warm-up

A single deopt is cheap (microseconds). The pathology is a deopt loop: optimize, deopt, re-optimize, deopt. V8 counts deopts per function and eventually gives up optimizing it, leaving it in a lower tier permanently. Common reasons in real traces:

[bailout (kind: deopt-eager, reason: not a Smi): ...]
[bailout (kind: deopt-eager, reason: out of bounds): ...]
[bailout (kind: deopt-lazy, reason: (unknown)): ...]
[bailout (kind: deopt-eager, reason: Insufficient type feedback for call): ...]

Reading deopt reasons

"wrong map" means a shape changed — look at object construction. "not a Smi" / "lost precision" means integers became floats or overflowed 31/32 bits. "out of bounds" means reading past an array's length (often a loop with <=). "Insufficient type feedback" during warm-up is normal; repeated ones in steady state are not. Fix patterns are in V8 Optimization.

Part 2: Objects, Shapes, and Inline Caches

Hidden Classes (Maps)

JavaScript objects look like hash maps, but treating them as hash maps would make every property access a hash lookup. V8 instead gives each object a pointer to a Map (called a hidden class or shape in other engines) that describes its layout. Objects built the same way share a Map.

flowchart LR
    M0["Map M0<br/>(empty)"] -->|"add 'x'"| M1["Map M1<br/>x @ offset 0"]
    M1 -->|"add 'y'"| M2["Map M2<br/>x @ 0, y @ 1"]
    M1 -->|"add 'z'"| M3["Map M3<br/>x @ 0, z @ 1"]
    M0 -->|"add 'y'"| M4["Map M4<br/>y @ 0"]
    M4 -->|"add 'x'"| M5["Map M5<br/>y @ 0, x @ 1"]

Maps form a transition tree. Adding a property follows (or creates) a transition edge. {x, y} and {y, x} end up on different Maps (M2 vs M5) even though they have the same keys, because property order is part of the layout.

What a Map stores:

FieldPurpose
Descriptor arrayProperty names, attributes, field index, field representation
Field representationSmi, Double, HeapObject, or Tagged per field
PrototypeSo prototype-chain lookups can be cached by Map
Instance size / in-object property countHow many properties live inside the object vs. a separate backing store
Elements kindHow indexed properties are stored (see below)
TransitionsEdges to child Maps
Stability / deprecation bitsWhether optimized code may depend on this Map not changing

In-object vs. out-of-object properties. V8 pre-allocates some slots directly inside the object. Properties beyond that go into a separate properties backing store (one extra pointer hop). For constructor functions and classes, V8 uses slack tracking: the first several instances are allocated generously, and once V8 has seen how many properties instances actually get, it shrinks the instance size for future allocations. This is why initializing all fields in the constructor matters — late-added fields often land out-of-object.

Field representation generalization. If field x has always held Smis, the Map records Smi and optimized code can load it as an integer without checks. Store a float into it and the field generalizes to Double (stored unboxed or as a HeapNumber); store an object and it becomes Tagged. Generalization deprecates the old Map and lazily deoptimizes code that depended on it. Generalization only ever goes one way, so a single stray assignment permanently weakens the field for every object of that shape.

Dictionary mode. When an object gets too many properties added dynamically, or you delete a property that is not the last one added, V8 converts it to dictionary mode: a real hash table with its own Map that says "slow." Property access works but no inline cache can use a fixed offset. Objects used as hash maps (cache[key] = value with arbitrary keys) are in dictionary mode by design — which is fine, and also why Map is usually a better choice for that job.

You can inspect all of this with natives syntax:

// node --allow-natives-syntax shapes.js
class Point { constructor(x, y) { this.x = x; this.y = y; } }
const a = new Point(1, 2);
const b = new Point(3, 4);
console.log(%HaveSameMap(a, b));   // true

const c = new Point(1, 2);
c.z = 3;                            // transition to a child Map
console.log(%HaveSameMap(a, c));   // false

const d = new Point(1.5, 2);       // x generalizes Smi -> Double
console.log(%HaveSameMap(a, d));   // true again: a's Map was deprecated and migrated
%DebugPrint(a);                     // prints map, fields, representations

Inline Caches

An inline cache (IC) is the feedback slot plus a fast-path handler for one operation site. Each IC moves through states:

stateDiagram-v2
    [*] --> Uninitialized
    Uninitialized --> Monomorphic: first execution, 1 map
    Monomorphic --> Polymorphic: 2nd distinct map
    Polymorphic --> Polymorphic: up to 4 maps
    Polymorphic --> Megamorphic: 5th distinct map
    Monomorphic --> Megamorphic: repeated map deprecation
    Megamorphic --> [*]
StateMaps seenLookup costOptimizer output
Monomorphic1One map compare + load at fixed offsetCheckMaps + direct load; can inline
Polymorphic2-4Linear compare against each mapBranch per map; still inlinable
Megamorphic5+Global stub cache (hashed by map and name), falls back to full lookup on missGeneric call to a load builtin; no inlining

A megamorphic site is not catastrophically slow in isolation — the global stub cache is a reasonably fast hash. The real cost is what it blocks: the optimizer cannot inline, cannot eliminate checks, and cannot do escape analysis through it. In tight loops this is often 5-20x versus monomorphic, and the effect compounds through every function that calls it.

// One site, many shapes -> megamorphic
function getId(entity) { return entity.id; }

getId({ id: 1 });
getId({ id: 2, name: 'a' });
getId({ name: 'b', id: 3 });            // different order -> different map
getId(new User(4));
getId(new Order(5));                    // 5 maps: megamorphic

// Monomorphic alternative: normalize shape at the boundary
class EntityRef { constructor(id, kind) { this.id = id; this.kind = kind; } }

The classic real-world source of megamorphism is a generic utility (get(obj, path), an ORM's row.toJSON(), a logging serializer) that sees every shape in the program at one property-access site. The fix is either shape normalization upstream or accepting that the generic utility lives off the hot path.

Elements Kinds

Indexed properties (arr[0]) are stored separately from named properties, in an elements backing store. The Map records the elements kind, which lets optimized code skip type checks per element.

flowchart LR
    PS["PACKED_SMI_ELEMENTS"] --> PD["PACKED_DOUBLE_ELEMENTS"]
    PD --> PE["PACKED_ELEMENTS"]
    PS --> HS["HOLEY_SMI_ELEMENTS"]
    PD --> HD["HOLEY_DOUBLE_ELEMENTS"]
    PE --> HE["HOLEY_ELEMENTS"]
    HS --> HD
    HD --> HE
    HE --> DICT["DICTIONARY_ELEMENTS<br/>(sparse)"]

Transitions only go down and right — toward more general. There is no way back for that array.

OperationResulting kindWhy it matters
[1, 2, 3]PACKED_SMIElements are raw tagged Smis; fastest
arr.push(1.5)PACKED_DOUBLEStored as unboxed float64 array; still fast
arr.push('a')PACKED_ELEMENTSEach element is a tagged pointer; type check per read
new Array(3)HOLEY_SMIHoles must check the prototype chain on read
arr[1000] = 1 on short arrayDICTIONARYHash table; very slow iteration
arr.push(-0) or NaN into Smi arrayPACKED_DOUBLE-0 and NaN are not Smis

Holes are worse than they look: reading a hole means "not present here, check Array.prototype and Object.prototype," so the optimizer must either insert a prototype-chain check or depend on the prototypes being pristine (a protector — a global flag that, if invalidated by someone patching Array.prototype, lazily deopts every dependent function in the process).

Typed arrays sidestep all of this

Float64Array, Int32Array, and friends have a fixed element type and no holes. They are backed by an ArrayBuffer outside the normal elements store, so numeric hot paths (parsers, image processing, simulations) avoid elements-kind transitions entirely. They also cross to native code without copying, which matters for addons (see Part 4).

Value Representation: Smis, HeapNumbers, and Pointer Compression

Every JS value in V8 is a tagged word. The low bit distinguishes a Smi (small integer, stored inline) from a pointer to a heap object. Floats, large integers, strings, and objects are all heap objects.

BuildTagged wordSmi rangeNotes
64-bit, no pointer compression (Node default)64 bits32-bit signedPointers are full 64-bit
64-bit with pointer compression (Chrome default)32 bits31-bit signedHeap confined to a 4 GB "cage"; ~40% less heap memory

Node.js builds without pointer compression by default, so its heap is not limited to 4 GB per isolate — but each pointer costs 8 bytes rather than 4. If you compare memory use between Chrome and Node for the same code, this is a significant part of the difference.

A number that is not a Smi — 0.5, 2 ** 40, -0 — is boxed as a HeapNumber unless the optimizer can keep it unboxed in a register or a Double-represented field. Arithmetic-heavy code that keeps "accidentally" producing floats allocates on every operation in unoptimized tiers.

Part 3: Memory and the Orinoco Garbage Collector

Heap Layout

flowchart TB
    subgraph HEAP["V8 heap (per isolate)"]
        subgraph YG["Young generation"]
            NURS["Semi-space A<br/>(allocation)"]
            SURV["Semi-space B<br/>(copy target)"]
        end
        subgraph OG["Old generation"]
            OLD["Old space<br/>(survived objects)"]
            CODE["Code space<br/>(JIT output)"]
            LO["Large object space<br/>(objects above ~ page size)"]
            TR["Trusted / read-only spaces"]
        end
    end
    EXT["External memory<br/>(ArrayBuffer backing stores,<br/>Buffer, native addon memory)"]
    HEAP -.->|"tracked by size,<br/>not by GC tracing"| EXT

V8's collector is built on the generational hypothesis: most objects die young. Request-scoped objects (parsed JSON, intermediate arrays, closures for a promise chain) usually die before the request finishes. A few objects (caches, module state, connection pools) live forever.

SpaceCollected byTypical sizeAlgorithm
Young generationScavenger (minor GC)Semi-spaces of a few MB to 16 MB+ each (--max-semi-space-size)Parallel copying (Cheney-style)
Old spaceMark-compact (major GC)Up to --max-old-space-sizeConcurrent mark, parallel compact, concurrent sweep
Large object spaceMajor GCObjects too big for a normal pageNever moved; marked and freed
Code spaceMajor GCJIT outputMark-compact, with flushing of old bytecode

The Scavenger (Minor GC)

sequenceDiagram
    participant M as Main thread
    participant H as Helper threads
    participant FS as From-space
    participant TS as To-space / Old space

    Note over M: Allocation fails: young gen full
    M->>M: Pause JS
    M->>H: Distribute roots<br/>(stack, globals, remembered set)
    par Parallel scavenge
        H->>FS: Find live object
        H->>TS: Copy to to-space<br/>(or promote to old space if it survived once)
        H->>FS: Leave forwarding pointer
    end
    M->>M: Swap semi-spaces#59;<br/>from-space is now empty
    Note over M: Resume JS (pause: ~0.5-5 ms)

Allocation in the young generation is a pointer bump — just as fast as a stack allocation. Collection cost is proportional to the live objects, not the dead ones; garbage is free. That is why allocating many short-lived objects is usually fine, while allocating objects that survive one scavenge and then die (promoted, then garbage in old space) is expensive: they pay for a copy and then burden the major GC.

Write barriers and the remembered set. To collect the young generation without scanning the whole old generation, V8 must know which old objects point to young ones. Every pointer store in JIT code includes a write barrier: a few instructions that, if an old object is made to point at a young object, record the slot in a remembered set. Those slots become extra roots for the scavenger. The barrier is also used by concurrent marking (below). It is part of why storing pointers is more expensive than storing numbers.

Mark-Compact (Major GC)

Major GC has three phases, and Orinoco's contribution (2016-2019) was making each one concurrent, parallel, or incremental so that the main thread pauses as little as possible.

flowchart LR
    subgraph MARK["1. Marking"]
        IM["Incremental steps<br/>on main thread"]
        CM["Concurrent marking<br/>on helper threads"]
        FIN["Final atomic pause:<br/>re-scan roots, finish worklist"]
    end
    subgraph SWEEP["2. Sweeping"]
        CS["Concurrent sweeping:<br/>build free lists"]
    end
    subgraph COMPACT["3. Compaction"]
        PC["Parallel evacuation<br/>of fragmented pages"]
        UP["Update pointers"]
    end
    IM --> FIN
    CM --> FIN
    FIN --> CS
    FIN --> PC
    PC --> UP

Marking uses the tri-color abstraction: white (not yet seen), grey (seen, children not yet scanned), black (fully scanned). Helper threads drain grey worklists while JS keeps running. The danger is that JS creates a pointer from a black object to a white object behind the marker's back; the marking write barrier catches that store and greys the target. A short final pause rescans the stack and finishes leftovers.

Sweeping walks pages and builds free lists from dead memory. It runs concurrently; the allocator can lazily sweep a page on demand if it needs space before the sweeper reaches it.

Compaction moves objects out of highly fragmented pages so those pages can be released. It is done in parallel during a pause, and only for pages where fragmentation makes it worth it.

TechniqueWhat runs on the main threadEffect
Stop-the-worldEverythingSimple, long pauses (hundreds of ms on large heaps)
ParallelEverything, but helpers speed it upShorter pauses, still a pause
IncrementalSmall slices interleaved with JSMany tiny pauses; total work slightly higher
ConcurrentAlmost nothing; helpers work while JS runsLowest pause, needs barriers, competes for CPU cores

Containers and helper threads

Concurrent GC assumes spare cores. In a container limited to one CPU (cpus: 1 or a CFS quota of 100ms/100ms), "concurrent" marking competes with your JS thread for the same core, and parallel scavenges cannot actually run in parallel. GC pauses that are invisible on a laptop become visible in production. Give Node services at least 2 vCPUs or measure GC time explicitly with --trace-gc and perf_hooks.

Observing GC

node --trace-gc server.js
[48213:0x5d1f...]   2104 ms: Scavenge 38.2 (52.1) -> 31.0 (53.1) MB, pooled: 0 MB, 2.11 / 0.00 ms  (average mu = 0.991, current mu = 0.987) allocation failure;
[48213:0x5d1f...]   9871 ms: Mark-Compact (reduce) 212.4 (240.6) -> 120.7 (178.3) MB, 41.20 / 0.00 ms  (+ 85.3 ms in 112 steps since start of marking, biggest step 3.1 ms, walltime since start of marking 902 ms) (average mu = 0.934, current mu = 0.912) finalize incremental marking via task; GC in old space requested

Read it as: kind before (committed) -> after (committed) MB, main-thread pause ms. The mu (mutator utilization) is the fraction of time JS got to run; below ~0.9 sustained means GC is a real cost. The parenthetical "+ 85.3 ms in 112 steps" is incremental work spread across the marking window.

Programmatic tracking for dashboards:

import { PerformanceObserver, constants } from 'node:perf_hooks';

const kinds = {
  [constants.NODE_PERFORMANCE_GC_MINOR]: 'scavenge',
  [constants.NODE_PERFORMANCE_GC_MAJOR]: 'mark-compact',
  [constants.NODE_PERFORMANCE_GC_INCREMENTAL]: 'incremental',
  [constants.NODE_PERFORMANCE_GC_WEAKCB]: 'weak-callbacks',
};

const obs = new PerformanceObserver((list) => {
  for (const entry of list.getEntries()) {
    const kind = kinds[entry.detail.kind] ?? 'other';
    gcDurationHistogram.observe({ kind }, entry.duration);   // e.g. prom-client Histogram
  }
});
obs.observe({ entryTypes: ['gc'] });

Heap Limits and External Memory

FlagDefault (64-bit, recent Node)When to change
--max-old-space-size=<MB>Derived from system memory; in containers recent Node versions respect cgroup limitsSet to ~75% of container memory so the process GCs harder before the OOM killer acts
--max-semi-space-size=<MB>~16 MBRaise (32-64 MB) for allocation-heavy services to cut scavenge frequency and premature promotion; costs RSS
--expose-gcoffTests and benchmarks only

Buffer and ArrayBuffer contents live outside the V8 heap as external memory. V8 tracks their size (so a lot of external memory pressures GC to run), but --max-old-space-size does not bound it. A process with a 512 MB heap limit can have 2 GB RSS from buffers. When RSS and heapUsed diverge, check process.memoryUsage().external and arrayBuffers first. Leak-hunting technique is covered in Memory Management.

Part 4: libuv and the Event Loop

Architecture

libuv gives Node a single cross-platform abstraction over very different OS I/O models:

PlatformSocket readinessFile I/ONotes
LinuxepollThread pool (io_uring for some ops in newer libuv, gated in Node)Most production Node
macOS / BSDkqueueThread pool
WindowsIOCP (completion-based)IOCP / thread poolCompletion vs. readiness model hidden by libuv

The crucial split: network I/O is truly asynchronous at the OS level, so a single thread can wait on 100,000 sockets with one epoll_wait call. Regular file I/O on Linux is not (readiness APIs report regular files as always ready, and blocking reads can still stall on disk), so libuv runs it on a thread pool and reports completion back to the loop.

flowchart TB
    subgraph MAIN["Main thread"]
        JS["V8: your JS callbacks"]
        LOOP["uv_run() loop"]
    end
    subgraph POOL["libuv thread pool (default 4)"]
        T1["worker 1"]
        T2["worker 2"]
        T3["worker 3"]
        T4["worker 4"]
    end
    Q["Work queue (FIFO)"]
    K["Kernel: epoll/kqueue/IOCP"]
    AS["uv_async wakeup<br/>(eventfd / pipe)"]

    JS -->|"fs.readFile, crypto.pbkdf2,<br/>dns.lookup, zlib"| Q
    Q --> T1
    Q --> T2
    Q --> T3
    Q --> T4
    T1 -->|"done"| AS
    AS --> LOOP
    JS -->|"socket read/write"| K
    K -->|"readiness events"| LOOP
    LOOP --> JS

Event Loop Phases

flowchart TB
    START(["loop iteration"]) --> TIMERS["timers<br/>setTimeout / setInterval<br/>callbacks whose time has come"]
    TIMERS --> PENDING["pending callbacks<br/>I/O callbacks deferred<br/>from the previous iteration"]
    PENDING --> IDLE["idle, prepare<br/>(internal)"]
    IDLE --> POLL["poll<br/>wait for I/O (epoll_wait)<br/>run I/O callbacks"]
    POLL --> CHECK["check<br/>setImmediate callbacks"]
    CHECK --> CLOSE["close callbacks<br/>socket.on('close')"]
    CLOSE --> ALIVE{"active handles<br/>or requests?"}
    ALIVE -->|"yes"| START
    ALIVE -->|"no"| EXIT(["process exits"])
PhaseRunsBlocks?
timersExpired setTimeout/setInterval callbacks, ordered by expiryNo
pending callbacksSome system callbacks (e.g. certain TCP errors) deferred from the last pollNo
idle / preparelibuv internals; Node uses prepare to start the event-loop-delay timerNo
pollComputes timeout from the nearest timer, blocks in epoll_wait, runs I/O callbacksYes — this is where an idle process sleeps
checksetImmediate callbacksNo
close'close' events of handles closed with uv_closeNo

Microtasks are not a phase. After every callback that Node invokes from the loop (since Node 11), it drains the process.nextTick queue completely, then the promise microtask queue completely, repeating until both are empty. That has two consequences:

  1. A promise chain that keeps resolving synchronously (or recursive nextTick) starves the loop indefinitely — no I/O, no timers.
  2. A setImmediate scheduled from an I/O callback always runs before a setTimeout(fn, 0) scheduled from the same callback, because check comes before the next timers phase. From the main module the order is nondeterministic. Detailed ordering rules and examples are in Node.js Event Loop.

What Actually Uses the Thread Pool

This is the single most useful table on this page for production debugging:

APIMechanismUses thread pool?
net, http, https sockets, tlsReadiness via epoll/kqueue/IOCPNo
fs.* async and fs.promises (read, write, stat, open, readdir...)Blocking syscalls on pool threadsYes
fs.watchinotify / FSEvents / ReadDirectoryChangesWNo
dns.lookup() (default for http.get, net.connect by hostname)getaddrinfo(3) — blocking libc callYes
dns.resolve*(), dns.Resolverc-ares over socketsNo
crypto.pbkdf2, scrypt, randomBytes (async), randomFill, generateKeyPair, async hkdfPool workYes
crypto.createHash, createCipheriv, signSynchronous on main threadNo (but blocks JS!)
zlib async (gzip, deflate, brotliCompress)Pool workYes
child_processfork/spawn + pipesNo (pipes are polled)
TimersMin-heap in the loopNo
Node-API napi_create_async_workPool workYes

The default pool size is 4, set via the UV_THREADPOOL_SIZE environment variable (maximum 1024). It is sized once, the first time anything uses the pool, so it must be set before startup — setting process.env.UV_THREADPOOL_SIZE late in your code is unreliable.

sequenceDiagram
    participant R as 100 requests
    participant Q as Pool queue
    participant P as 4 pool threads
    participant F as fs.readFile (2 ms)

    R->>Q: 50 x crypto.pbkdf2 (100 ms each)
    R->>Q: 50 x fs.readFile
    Q->>P: threads busy with pbkdf2 for 1.25 s
    Note over F: readFile waits behind<br/>all 50 hash jobs:<br/>2 ms becomes ~1250 ms
    P->>F: finally scheduled

War Story (illustrative)

The DNS lookup that slowed down file uploads

A service accepted uploads, wrote them to local disk, and posted metadata to an internal API by hostname. A resolver change made getaddrinfo slow (~2 s per lookup on cache miss). Every outbound request occupied a pool thread for 2 seconds inside dns.lookup, and with only four threads, the fs.write calls for uploads queued behind them. Upload latency jumped from 40 ms to 6 seconds while CPU sat at 5%. Nothing in the JS profile was hot; the event loop delay histogram was flat. The clue was the thread pool: four threads, all inside getaddrinfo.

The fix was three lines: a caching lookup function passed to the HTTP agent, UV_THREADPOOL_SIZE=16, and a dashboard panel for pool queue depth. The lesson: "async" in Node does not mean "free" — it means "runs somewhere else that can also saturate."

Monitoring the Loop

import { monitorEventLoopDelay, performance } from 'node:perf_hooks';

const delay = monitorEventLoopDelay({ resolution: 10 });
delay.enable();

let lastElu = performance.eventLoopUtilization();

setInterval(() => {
  const elu = performance.eventLoopUtilization(lastElu);   // delta since last sample
  lastElu = performance.eventLoopUtilization();

  metrics.gauge('nodejs.eventloop.delay.p99_ms', delay.percentile(99) / 1e6);
  metrics.gauge('nodejs.eventloop.delay.max_ms', delay.max / 1e6);
  metrics.gauge('nodejs.eventloop.utilization', elu.utilization);  // 0..1
  delay.reset();
}, 10_000).unref();
SignalHealthyInvestigate
Event loop delay p99< 20 ms> 100 ms: something is blocking the JS thread (sync crypto, JSON of huge payloads, regex backtracking, GC)
Event loop utilization< 0.7> 0.9 sustained: the JS thread is the bottleneck; scale out or move work to Worker Threads
High latency, low ELU, low CPU—Waiting on something external: thread pool saturation, DB pool, downstream service

Part 5: Crossing the JS/Native Boundary

How a Core API Reaches C++

Calling fs.readFile is several layers deep:

sequenceDiagram
    participant U as Your JS
    participant L as lib/fs.js
    participant B as internalBinding('fs')<br/>(C++ in src/node_file.cc)
    participant UV as libuv
    participant T as Pool thread
    participant OS as Kernel

    U->>L: fs.readFile(path, cb)
    L->>L: validate args,<br/>create FSReqCallback
    L->>B: binding.open(path, flags, mode, req)
    B->>UV: uv_fs_open(loop, req, ..., AfterOpen)
    UV->>T: enqueue work
    T->>OS: open(2) — blocks this thread
    OS-->>T: fd
    T-->>UV: post completion
    UV-->>B: AfterOpen (on main thread)
    B-->>L: req.oncomplete(err, fd)
    L->>B: binding.read(...)  (and so on)
    L-->>U: cb(null, data)

Points that explain real behavior:

  • internalBinding is how Node's own JS reaches its C++ layer. The older public process.binding() is deprecated; do not depend on it.
  • fs.readFile is not one operation. It is open, fstat, one or more reads, and close — each a separate pool trip. For a large file, that is many round trips through the queue, which is why fs.readFile of a big file under pool contention is so much slower than fs.createReadStream with a sensible highWaterMark, or a single fs.promises.readFile on a quiet system.
  • Callbacks arrive on the main thread in the poll phase. The pool threads never touch V8; they cannot, because a V8 isolate is single-threaded.

Node-API and C++ Addons

There are three generations of addon interfaces:

InterfaceABI stable across Node versions?Status
Raw V8/libuv APIsNo — recompile for every major NodeAvoid for new code
NAN (Native Abstractions for Node)No — source-compatible macros onlyLegacy
Node-API (napi_*, C) + node-addon-api (C++ wrapper)YesUse this

Node-API is a C ABI that hides V8 entirely: you work with opaque napi_value handles, and a binary compiled against Node-API version N runs on every later Node release. That stability is why prebuilt binaries (prebuildify, node-gyp-build) work.

A synchronous function with node-addon-api:

// src/sum.cc
#include <napi.h>

Napi::Value SumFloat64(const Napi::CallbackInfo& info) {
  Napi::Env env = info.Env();
  if (info.Length() < 1 || !info[0].IsTypedArray() ||
      info[0].As<Napi::TypedArray>().TypedArrayType() != napi_float64_array) {
    Napi::TypeError::New(env, "expected Float64Array").ThrowAsJavaScriptException();
    return env.Null();
  }
  auto arr = info[0].As<Napi::Float64Array>();
  // Zero-copy: points directly into the ArrayBuffer's backing store
  const double* data = arr.Data();
  size_t n = arr.ElementLength();

  double total = 0.0;
  for (size_t i = 0; i < n; ++i) total += data[i];
  return Napi::Number::New(env, total);
}

Napi::Object Init(Napi::Env env, Napi::Object exports) {
  exports.Set("sumFloat64", Napi::Function::New(env, SumFloat64));
  return exports;
}

NODE_API_MODULE(sum, Init)

An asynchronous one that runs on the libuv pool. The key rule: Execute() runs on a pool thread and must not touch any JS value; copy inputs out first and build JS results in OnOK() back on the main thread.

// src/hash_async.cc
#include <napi.h>
#include <string>
#include <vector>

class HashWorker : public Napi::AsyncWorker {
 public:
  HashWorker(Napi::Function& cb, std::vector<uint8_t> input, uint32_t rounds)
      : Napi::AsyncWorker(cb), input_(std::move(input)), rounds_(rounds) {}

  // Pool thread: no Napi::Value, no V8, no exceptions to JS
  void Execute() override {
    if (rounds_ == 0) { SetError("rounds must be > 0"); return; }   // rejects in OnError
    uint64_t h = 1469598103934665603ULL;            // FNV-1a offset basis
    for (uint32_t r = 0; r < rounds_; ++r) {
      for (uint8_t b : input_) { h ^= b; h *= 1099511628211ULL; }
    }
    result_ = h;
  }

  // Main thread: safe to create JS values
  void OnOK() override {
    Napi::Env env = Env();
    Callback().Call({ env.Null(), Napi::BigInt::New(env, result_) });
  }

 private:
  std::vector<uint8_t> input_;
  uint32_t rounds_;
  uint64_t result_ = 0;
};

Napi::Value HashAsync(const Napi::CallbackInfo& info) {
  auto buf = info[0].As<Napi::Buffer<uint8_t>>();
  uint32_t rounds = info[1].As<Napi::Number>().Uint32Value();
  Napi::Function cb = info[2].As<Napi::Function>();
  // Copy: the Buffer could be mutated or collected while the worker runs
  std::vector<uint8_t> copy(buf.Data(), buf.Data() + buf.Length());
  (new HashWorker(cb, std::move(copy), rounds))->Queue();   // self-deleting
  return info.Env().Undefined();
}
{
  "targets": [{
    "target_name": "native",
    "sources": ["src/sum.cc", "src/hash_async.cc"],
    "include_dirs": ["<!@(node -p \"require('node-addon-api').include\")"],
    "defines": ["NAPI_VERSION=8", "NAPI_DISABLE_CPP_EXCEPTIONS"],
    "cflags_cc": ["-O3"]
  }]
}

The Cost of Crossing

Every call into native code pays for argument checking, handle creation, and conversion. Rough orders of magnitude on a modern x86 server:

CallApproximate overheadImplication
JS -> JS (inlined)~0 nsThe optimizer erased the call
JS -> JS (not inlined)2-5 ns
JS -> Node-API function20-100 nsFine for coarse calls, ruinous per element
Passing a string to nativeO(length): UTF-16 -> UTF-8 copyPass Buffer/TypedArray for large data
Passing a TypedArray~constant: pointer to backing storeZero-copy
Async work round trip10-50 µs + pool queueingOnly worth it for work > ~100 µs

V8 has a Fast API mechanism that lets the optimizer call certain C++ functions directly with unboxed arguments, bypassing most of this overhead; Node uses it internally for hot bindings (some Buffer, perf_hooks, and url paths). It is not part of the stable Node-API surface for addon authors.

When native is worth it

Native code wins when the work per call is large and the data crosses as typed arrays: image codecs, compression, crypto primitives, numeric kernels. It loses when called per item in a hot loop with strings or objects — the boundary cost exceeds the speedup, and you also lose inlining. Before writing C++, try a Worker Thread (for parallelism) or WebAssembly (for portable compute); see WebAssembly.

Threads, Isolates, and Workers

A V8 isolate is an independent heap plus its own compiler state; it may only be entered by one thread at a time. worker_threads creates a new isolate and a new libuv loop on a new OS thread — which is why workers do not share JS objects and communicate by structured clone or SharedArrayBuffer. All workers in a process share the one libuv thread pool, so moving pool-heavy work into workers does not add pool capacity.

PrimitiveNew isolate?New event loop?Shares heap?Shares thread pool?
worker_threads.WorkerYesYesNo (only SharedArrayBuffer)Yes
child_process.forkYes (new process)YesNoNo
cluster workerYes (new process)YesNoNo
Async work in an addonNoNo—Yes (it is the pool)

Putting It Together: A Request's Life

sequenceDiagram
    participant K as Kernel
    participant L as libuv poll
    participant H as http parser (llhttp, C)
    participant J as Your handler (JIT code)
    participant G as GC
    participant P as Thread pool

    K->>L: socket readable (epoll)
    L->>H: on read callback
    H->>J: 'request' event -> handler
    J->>J: JSON.parse body<br/>(young-gen allocations)
    J->>P: crypto.scrypt(password)
    Note over J: returns to loop#59;<br/>microtasks drain
    G->>G: scavenge (1-2 ms pause)<br/>parse garbage freed for free
    P-->>L: scrypt done
    L->>J: callback -> respond
    J->>K: socket write (may complete synchronously)

Each stage maps to a failure mode you can now name:

SymptomLayerFirst thing to check
Throughput drops after a deploy, no CPU changeV8 tiers--trace-deopt for deopt loops in hot functions
CPU high, flame graph shows LoadIC_Megamorphic / KeyedLoadIC_MegamorphicInline cachesA generic accessor seeing many shapes
Periodic latency spikes every few secondsGC--trace-gc, major GC pause, mutator utilization
RSS grows, heapUsed flatExternal memoryprocess.memoryUsage().arrayBuffers, native addon leaks
Latency high, CPU low, loop delay lowThread poolPool-bound APIs (dns.lookup, fs, crypto, zlib) and pool size
Latency high, loop delay highMain threadCPU profile for sync work

Diagnostic Flag Reference

FlagShows
--print-bytecode --print-bytecode-filter=fnIgnition bytecode for a function
--trace-opt / --trace-deoptTier-up decisions and deopt reasons
--log-icWrites IC state transitions to the V8 log (process with v8-tools / Indicium)
--trace-mapsMap creation and transitions (very verbose)
--allow-natives-syntaxEnables %DebugPrint, %HaveSameMap, %GetOptimizationStatus, %OptimizeFunctionOnNextCall
--no-opt, --no-maglev, --no-sparkplugDisable tiers to isolate a JIT-related difference
--trace-gc, --trace-gc-verboseEvery GC with pause times
--max-old-space-size, --max-semi-space-sizeHeap sizing (MB)
--prof + node --prof-processSampling profile including VM-internal ticks (GC, IC misses, compiler)
--cpu-prof, --heap-prof.cpuprofile / .heapprofile for DevTools

Natives syntax is for investigation only

% functions are internal, change between V8 versions, and can crash the process with bad arguments. Use them in scratch scripts to understand behavior; never ship them, and never make performance decisions from a microbenchmark that forces optimization with %OptimizeFunctionOnNextCall without also checking real-traffic behavior.

Further Reading


"Every rule about fast JavaScript is a sentence about an engine assumption. Learn the assumptions and you will not need the rules."