On this page

Operating Systems Fundamentals

OS concepts are the substrate everything else runs on. When you understand how the kernel manages processes, memory, and storage, system design decisions that seemed arbitrary become obvious — why connection pools exist, why context switching is expensive, why fork() is fast even for large processes, why mmap beats read() for large files.

These topics appear in FAANG SDE interviews at the senior level: "explain what happens when you call fork()", "how does the kernel schedule threads on a multi-core machine?", "what is a page fault?", "why is reading a file from disk so much slower than reading from memory?"

But the bigger reason to learn this material is not interviews. It's that a large share of hard production incidents — the ones where the dashboards look fine and the service is still slow, or where data is lost even though every write "succeeded" — are explained one layer below the application, in the kernel. This section is written to make that layer readable.


What the OS Does

The operating system is a resource manager. It mediates access to:

  • CPU time — scheduling which process runs when
  • Memory — giving each process the illusion of owning all memory via virtual address spaces
  • Storage — file systems that map names to data on disk
  • Devices — abstracted via device drivers

It also provides two things that aren't resources but are just as fundamental:

  • Isolation and protection — one process can't read another's memory or files without permission; containers are built from these same mechanisms
  • Communication — pipes, sockets, signals, shared memory and the network stack, so isolated processes can still cooperate

The kernel runs in privileged mode (ring 0). User programs run in unprivileged mode (ring 3). Crossing the boundary requires a system call — a controlled gate into the kernel.

User Space:   your_program → libc → system call interface
─────────────────────────────────────────────────────────
Kernel Space:  syscall handler → scheduler / memory mgr / VFS / drivers
flowchart TB
    subgraph US["User space (ring 3)"]
        APP["Your service: Go, Java, Node, Python"]
        RT["Language runtime: GC, goroutine or virtual-thread scheduler, event loop"]
        LIBC["libc / syscall wrappers / vDSO"]
    end
    subgraph KS["Kernel space (ring 0)"]
        SC["System call interface"]
        PROC["Processes and scheduling"]
        MEM["Virtual memory and page cache"]
        VFS["VFS and file systems"]
        NET["Networking stack"]
        SYNC["Locks, RCU, futexes"]
        CG["Namespaces and cgroups"]
        DRV["Drivers, interrupts, DMA"]
    end
    HW["Hardware: CPUs, RAM, NVMe, NIC, timers"]
    APP --> RT --> LIBC --> SC
    SC --> PROC
    SC --> MEM
    SC --> VFS
    SC --> NET
    PROC --> SYNC
    MEM --> DRV
    VFS --> MEM
    NET --> DRV
    CG -.->|"limits"| PROC
    CG -.->|"limits"| MEM
    DRV --> HW

Every page in this section zooms into one box of that diagram.


Why This Matters for System Design

Connection pools exist because thread creation is expensive (~1ms) and context switching has overhead (~1–10μs). Understanding this makes "why use a pool?" obvious rather than cargo-culted. (A bare pthread_create is ~10–20μs; the ~1ms figure is closer to establishing a new database connection or spawning a process — either way, you don't want to pay it per request.)

fork() is fast because virtual memory uses copy-on-write — the child shares the parent's pages until it writes to them. A 1GB process forks in milliseconds rather than the seconds a full copy would take: the kernel copies only page tables (~2MB per GB of resident memory).

mmap beats read() for large files because mmap maps file pages directly into the process address space — no copy to a user-space buffer. The kernel's page cache becomes the file buffer. (It isn't a universal win: page faults, TLB pressure and unpredictable I/O stalls are why many databases still prefer explicit reads.)

epoll beats select because epoll uses kernel data structures that don't scale linearly with the number of watched file descriptors.

SSDs vs HDDs — understanding that SSDs have ~0.1ms latency (vs 5–10ms for HDD) and that both are orders of magnitude slower than RAM (~100ns) gives intuition for caching, buffering, and when disk I/O is the bottleneck.

Production incidents that are really OS problems

What you seeWhat's actually happeningWhere to read
p99 latency spikes of ~50–100ms in Kubernetes while CPU usage is "well under the limit"CFS bandwidth throttling: the container burns its 100ms-period quota in a short multi-threaded burst, then every thread is paused until the next periodCPU Scheduling, Namespaces & cgroups
Pods restart with exit code 137 and no stack trace137 = 128 + SIGKILL: the cgroup memory limit was hit and the kernel's OOM killer terminated the processNamespaces & cgroups, Virtual Memory
Redis stalls for hundreds of ms every time it snapshotsfork() of a large process copies page tables while the event loop is paused; Transparent Huge Pages turn every post-fork write into a 2MB copyProcesses & Threads
A database lost acknowledged writes after a disk errorfsync() error semantics: after a writeback failure, Linux may drop the dirty pages and a retried fsync can succeed — PostgreSQL's 2018 "fsyncgate" changed it to panic instead of retryingPage Cache & fsync, File Systems
Too many open files under loadulimit -n defaults to 1,024 soft; every socket is a file descriptorFile Systems, Syscalls & Kernel Internals
Load average of 60 on a machine with idle CPUsTasks stuck in uninterruptible sleep (D state) on slow disk or NFS count toward loadProcesses & Threads, I/O & Storage
Deploys always take exactly 30 seconds; shutdown is never gracefulThe app is PID 1 in its container; the kernel doesn't apply default signal actions to PID 1, so SIGTERM is ignored until SIGKILLSignals & Process Control, Processes & Threads
Containers fail with fork: Resource temporarily unavailableZombie accumulation or thread explosion hitting pids.maxProcesses & Threads
A benchmark shows 5GB/s "disk" readsThe data was in the page cache; the disk was never touchedPage Cache & fsync
Packet drops and high ksoftirqd CPU at high request ratesSoftirq processing can't keep up; RX queues, RSS/IRQ affinity and NAPI budgets matterLinux Networking Stack, Interrupts, DMA & Drivers
Throughput collapses when adding threads; perf shows time in spinlocksLock contention in the kernel or a shared futexKernel Synchronization
Container escape or privilege escalation finding in a security reviewMissing seccomp profile, excess capabilities, running as root without user namespacesProcess Security

War Story: the 2012 leap second

On 30 June 2012 a leap second was inserted at midnight UTC. A bug in the Linux kernel's high-resolution timer handling left timers firing early, so applications sleeping on futexes with timeouts — notably Java services and MySQL — woke immediately, went back to sleep, and woke again, in a loop. CPU usage jumped to 100% across large parts of the internet (Reddit, Mozilla, Gawker and others reported outages or degradation). The widely used fix was simply to reset the clock (date -s "$(date)"), which re-armed the timers. Nothing in the application code was wrong; the failure lived entirely in the timekeeping and futex layer.


Six Mental Models That Explain Most of the Kernel

Before diving into individual pages, it helps to hold a few ideas that recur everywhere. Most "weird" kernel behavior is one of these playing out.

1. Everything is a trade between latency and throughput

Batching makes systems efficient; batching also makes individual operations wait. The scheduler's time slice (longer slices = less switching overhead, worse responsiveness), the page cache's writeback delay (dirty pages sit in RAM for up to ~30 seconds so writes can be merged), NIC interrupt coalescing, TCP's Nagle algorithm and io_uring's submission batching are all the same dial. When you tune any of them, ask which side of the trade you're moving toward and whether your workload wants that.

2. Virtualize, then share lazily

The kernel gives every process the illusion of its own CPU (scheduling), its own memory (virtual address spaces), its own files (fd tables) and, in containers, its own machine (namespaces). It then backs those illusions with shared physical resources and does real work only when forced: memory is allocated on first touch, fork() copies pages only when written, file data is read only when accessed and cached afterwards. "Lazy" is why the system is fast on average — and why costs show up later, in surprising places (a page-fault storm after fork, an OOM kill long after the malloc that "succeeded").

3. The boundary crossing is the cost

Ring 3 → ring 0 transitions, user ↔ kernel copies, and CPU ↔ device round trips are where time goes. Every major performance technique in this section reduces crossings: the vDSO (no syscall for time), epoll (one syscall for many sockets), io_uring (shared rings, often zero syscalls per I/O), sendfile/splice (no user-space copy), futexes (no syscall when a lock is uncontended), and kernel-bypass networking (no kernel at all).

4. Caches everywhere, and every cache can lie

CPU caches, the TLB, the page cache, the dentry/inode caches, the drive's own write cache. They make the common case fast and make performance non-linear: the second run of a benchmark is 100× faster, a process that migrates between CPUs suddenly slows down, a write that "completed" is actually in volatile memory. Correctness questions (durability) and performance questions (warm vs cold) both start with "which cache is this in?"

5. Limits are layered, and the smallest one wins

A thread can fail to be created because of ulimit -u, pids.max, threads-max, pid_max, vm.max_map_count or memory. A container's CPU can be capped by the cgroup quota, the CPU set, the node's capacity or its weight under contention. When something fails with EAGAIN or ENOMEM, list every layer of limit and check each — the error message rarely tells you which one fired.

6. The kernel optimizes for the average; you are paged for the tail

Default settings are tuned for a broad mix of workloads. Your p99.9 lives in the cases those defaults sacrifice: a burst that trips throttling, a writeback storm, a long kernel operation on a non-preemptible kernel, an unlucky NUMA placement. That's why the observability sections across these pages focus on histograms (run-queue latency, I/O latency) rather than averages.

flowchart LR
    LT["Latency vs throughput"] --> TUNE["Every tunable"]
    LAZY["Virtualize, share lazily"] --> SURPRISE["Costs appear later"]
    CROSS["Boundary crossings"] --> FAST["Most optimizations"]
    CACHE["Caches lie"] --> DUR["Durability and benchmark bugs"]
    LIMIT["Layered limits"] --> ERR["EAGAIN / ENOMEM puzzles"]
    TAIL["Tail vs average"] --> HIST["Measure histograms"]

Section Map

The section has 16 pages, grouped by the part of the kernel they cover.

Execution: running code

PageWhat it covers
Processes & Threadstask_struct, fork/exec/wait/exit, zombies, clone flags, thread models (1:1, M:N, goroutines, virtual threads), thread-pool sizing
CPU SchedulingTime quanta with worked numbers, FCFS/SJF/RR/MLFQ, CFS and EEVDF, real-time classes, context-switch cost, cgroup CPU throttling
Signals & Process ControlSIGTERM vs SIGKILL, async-signal safety, graceful shutdown
Syscalls & Kernel InternalsSyscall cost, vDSO, seccomp, eBPF, KPTI overhead
Interrupts, DMA & DriversHard and soft IRQs, NAPI, DMA, how devices talk to the kernel
Boot ProcessFirmware → bootloader → kernel → init/systemd, and how containers and microVMs shortcut it

Memory

PageWhat it covers
Virtual MemoryAddress spaces, page tables, TLB, page faults, copy-on-write, mmap
Memory AllocatorsHow malloc works, buddy/slab, fragmentation, jemalloc vs tcmalloc
Page Cache & fsyncBuffered vs direct I/O, dirty page writeback, fsync semantics and durability

Storage and I/O

PageWhat it covers
File SystemsInodes, VFS, journaling, what happens when you open() a file
I/O & Storage Internalsepoll vs select, io_uring, DMA, disk scheduling, zero-copy
Linux Networking StackPacket path from NIC to socket, TCP buffers, backlog queues, tuning

Coordination and isolation

PageWhat it covers
Kernel SynchronizationSpinlocks, mutexes, RCU, futexes, memory barriers
Inter-Process CommunicationPipes, Unix sockets, shared memory, message queues — and their costs
Namespaces & cgroupsHow containers work at the kernel level, OOM killer, CPU throttling
Process SecurityUsers, capabilities, seccomp, LSMs (SELinux/AppArmor), sandboxing
flowchart LR
    PT["Processes and Threads"] --> CS["CPU Scheduling"]
    PT --> VM["Virtual Memory"]
    PT --> SIG["Signals"]
    PT --> IPC["IPC"]
    CS --> KS["Kernel Synchronization"]
    VM --> MA["Memory Allocators"]
    VM --> PC["Page Cache and fsync"]
    PC --> FS["File Systems"]
    FS --> IO["I/O and Storage"]
    IO --> IRQ["Interrupts, DMA, Drivers"]
    IRQ --> NETS["Linux Networking Stack"]
    SC["Syscalls and Kernel"] --> SEC["Process Security"]
    PT --> NC["Namespaces and cgroups"]
    NC --> SEC
    BOOT["Boot Process"] -.-> PT

Learning Path

OrderPageKey Questions
1Processes & ThreadsWhat is a process? How does fork() work? Threads vs processes?
2CPU SchedulingHow does the kernel decide who runs next? What is context switching?
3Virtual MemoryWhat is a page fault? How does fork() not copy all memory?
4File SystemsWhat is an inode? What happens when you open() a file?
5I/O & Storage Internalsepoll vs select, io_uring, DMA, disk scheduling, zero-copy
6Memory AllocatorsHow malloc works, buddy/slab, fragmentation, jemalloc vs tcmalloc
7Signals & Process ControlSIGTERM vs SIGKILL, async-signal safety, graceful shutdown
8Namespaces & cgroupsHow containers work at the kernel level, OOM killer, CPU throttling
9Syscalls & Kernel InternalsSyscall cost, vDSO, seccomp, eBPF, KPTI overhead
10Kernel SynchronizationSpinlock vs mutex? What is RCU? How does a futex avoid syscalls?
11Inter-Process CommunicationPipe vs Unix socket vs shared memory — which is fastest and why?
12Page Cache & fsyncWhen is a write actually durable? Why did fsyncgate matter?
13Interrupts, DMA & DriversWhat happens when a packet or disk completion arrives?
14Linux Networking StackWhat does a packet go through between the NIC and recv()?
15Process SecurityCapabilities, seccomp, LSMs — how is a process confined?
16Boot ProcessWhat runs between power-on and PID 1?

Paths by goal

You don't have to read in order. Pick the path that matches why you're here:

GoalPath
Interview prep (senior backend)Processes & Threads → CPU Scheduling → Virtual Memory → File Systems → I/O & Storage → Syscalls
Running services on KubernetesProcesses & Threads → Signals → Namespaces & cgroups → CPU Scheduling (throttling section) → Virtual Memory (OOM) → Process Security
Databases and storage enginesVirtual Memory → Page Cache & fsync → File Systems → I/O & Storage → Memory Allocators → Kernel Synchronization
High-throughput networkingSyscalls → I/O & Storage (epoll, io_uring) → Interrupts, DMA & Drivers → Linux Networking Stack → CPU Scheduling (affinity, NUMA)
Performance debugging / SRECPU Scheduling (observability) → Virtual Memory → Page Cache & fsync → Syscalls (eBPF) → Kernel Synchronization

Numbers Every Engineer Should Know (OS Edition)

Order-of-magnitude figures for a modern x86-64 Linux server. Exact values vary 2–5× with hardware, kernel version, mitigations and virtualization — the point is which operations cost nanoseconds, microseconds or milliseconds. Measure on your own hardware before relying on any one figure.

OperationTypical costWhy it matters
L1 cache hit~1nsBaseline
L3 cache hit~10–20nsShared across cores on a socket
Main memory (local NUMA node)~80–100nsA pointer chase that misses cache
Main memory (remote NUMA node)~130–200nsWhy NUMA placement matters
clock_gettime() via vDSO~20nsNo kernel entry at all
Syscall round trip (no KPTI)~50–100nsgetpid-class calls
Syscall round trip with KPTI~200–500nsMeltdown mitigation cost
Minor page fault (page in RAM)~0.5–1μsFirst touch of malloced memory, CoW
Goroutine / green-thread switch~100–200nsUser-space scheduling
Thread context switch (same process)~1–2μsPlus cache/TLB refill afterwards
Process context switch~2–5μs direct, up to ~100μs indirectLarge working sets pay the refill
pthread_create + join~10–20μsWhy pools exist
fork() small process~50–100μsPage-table copy
fork() 1GB RSS process~10–20msPage tables scale with RSS
fork + exec + wait, small binary~0.5–1msProcess-per-request is expensive
Pipe / Unix socket round trip~2–10μsLocal IPC
TCP loopback round trip~10–30μsSame host, full TCP stack
Major page fault from NVMe~10–100μsPage cache miss on mmap
NVMe 4KB random read~20–100μsDevice latency
fsync on NVMe with power-loss protection~20–100μsEnterprise drives can ack from protected cache
fsync on consumer SSD~1–10msMust flush the drive cache
HDD seek + read~5–10msRotational latency
fsync on HDD~10–20msSeek + rotation
Scheduler tick1–4ms (HZ = 1000–250)Granularity of time-slice checks
Default fair-class slice (EEVDF base_slice)0.75ms base, ~3ms on 8+ CPUsHow long a CPU hog runs before others get a turn
SCHED_RR quantum100msReal-time round robin
cgroup CPU quota period100msThrottling can stall you for up to this long
Memory copy, single core~10–20 GB/s (~50–100μs per MB)Cost of extra copies (vs zero-copy)
Container start (runc, image cached)~100ms–1sMostly namespace + cgroup + filesystem setup
microVM boot (Firecracker)~125ms+Minimal kernel, minimal devices

Using these numbers

Turn them into ratios. One NVMe read ≈ 500 DRAM accesses ≈ 1,000 syscalls ≈ 20 context switches. A design that does one extra fsync per request on a consumer SSD has capped itself at a few hundred requests per second per disk before anything else is considered. A thread pool switching 100,000 times a second at ~5μs each spends half a core just switching.


The Toolbox: Which Tool Answers Which Question

QuestionFirst toolDeeper tool
Is the CPU saturated, and by whom?top / htop, mpstat -P ALL 1perf top, perf record -g
Are tasks waiting for CPU?vmstat 1 (r column), /proc/pressure/cpuperf sched latency, runqlat (bpftrace/bcc)
Is the container being throttled?cat /sys/fs/cgroup/cpu.statPrometheus container_cpu_cfs_throttled_periods_total
What is a process doing right now?strace -p PID, cat /proc/PID/wchanperf trace, offcputime
Memory pressure or leak?free -m, /proc/meminfo, /proc/pressure/memorysmem, /proc/PID/smaps, memleak
Disk the bottleneck?iostat -x 1biolatency, biosnoop
Network drops?ss -s, nstat, ip -s link/proc/net/softnet_stat, dropwatch, tcpretrans
Too many processes or threads?ps -eo nlwp,pid,comm --sort=-nlwppids.current in the cgroup
Why can't it open more files?cat /proc/PID/limits, ls /proc/PID/fd | wc -llsof -p PID
# The 60-second triage that answers most "is it the OS?" questions
uptime                 # load averages: trend up or down?
dmesg -T | tail        # OOM kills, hardware errors, hung tasks
vmstat 1 5             # r (run queue), b (blocked), si/so (swap), cs (switches)
mpstat -P ALL 1 3      # one hot CPU? high %iowait or %soft?
pidstat 1 3            # which processes use CPU
iostat -xz 1 3         # disk utilization and latency (r_await, w_await)
free -m                # memory and page cache
sar -n DEV,TCP 1 3     # network throughput, TCP retransmits
top                    # sanity check

(This is the shape of Brendan Gregg's well-known "Linux performance analysis in 60 seconds" checklist.)


System Call Reference (Linux)

CategoryKey Syscalls
Processfork(), exec(), exit(), wait(), getpid()
Memorymmap(), munmap(), brk(), mprotect()
File I/Oopen(), read(), write(), close(), lseek(), stat()
Socketssocket(), bind(), listen(), accept(), connect()
Signalskill(), signal(), sigaction(), pause()
IPCpipe(), msgget(), shmget(), semget()
Threads & syncclone(), clone3(), futex(), sched_yield(), set_tid_address()
Schedulingsched_setscheduler(), sched_setattr(), sched_setaffinity(), nice()
Durabilityfsync(), fdatasync(), sync_file_range(), msync()
Event I/Oepoll_create1(), epoll_ctl(), epoll_wait(), io_uring_setup(), io_uring_enter()
Isolationunshare(), setns(), pivot_root(), prctl(), seccomp()

Every time your Python code calls db.query(), your Go code calls http.Get(), or your Node.js event loop wakes up — a system call crosses the user/kernel boundary.

You can see exactly which ones, and how often:

$ strace -c -f curl -s https://example.com > /dev/null
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 31.20    0.000412          10        41           mmap
 18.03    0.000238          13        18           openat
 12.42    0.000164           8        20           read
  9.84    0.000130          21         6           poll
...
100.00    0.001320                   212        12 total

(Illustrative output — a single curl makes a couple of hundred syscalls, most of them during process startup and library loading.)


Interview Questions This Section Prepares You For

QuestionShort answerFull treatment
"What happens when you type ls and press Enter?"Shell fork()s, child execve()s /bin/ls, kernel maps the ELF, ld.so loads libc, ls makes getdents64/write syscalls, exits, shell wait()sProcesses & Threads
"Process vs thread?"Separate vs shared address space; in Linux both are tasks created by clone() with different sharing flagsProcesses & Threads
"What is a time quantum and how big should it be?"Max continuous run time before preemption; too small wastes CPU on switches, too large becomes FCFS; typically 1–10msCPU Scheduling
"How does Linux pick the next thread?"Fair class: vruntime weighted by nice; EEVDF (6.6+) picks the earliest virtual deadline among eligible tasksCPU Scheduling
"What is a page fault? Minor vs major?"Access to an unmapped/protected page; minor = page in RAM, major = must read from diskVirtual Memory
"When is a write durable?"After fsync/fdatasync succeeds on a drive that honours flushes — not after write() returnsPage Cache & fsync
"Why is epoll faster than select?"Interest list kept in the kernel; readiness delivered without scanning every fd each callI/O & Storage
"Spinlock or mutex?"Spin when the hold time is shorter than a sleep/wake cycle and you can't sleep; otherwise blockKernel Synchronization
"Fastest way for two processes to share data?"Shared memory (no copies) plus a synchronization primitive; Unix sockets when you want simplicityIPC
"What is a container, really?"A process tree with its own namespaces, constrained by cgroups, on the host kernelNamespaces & cgroups
"How does a packet get from the wire to recv()?"NIC DMA → IRQ → NAPI softirq → IP/TCP → socket buffer → wakeupLinux Networking Stack
"SIGTERM vs SIGKILL?"SIGTERM can be caught for graceful shutdown; SIGKILL cannotSignals

  • Concurrency — mutexes, deadlocks, lock-free structures and async patterns built on the primitives in this section
  • Networking — protocols above the kernel's networking stack
  • Databases — storage engines that live and die by page cache, fsync and allocator behavior
  • Distributed Systems — what happens when "the OS" becomes many machines