On this page

Math Patterns in System Design

System design interviews are not about memorizing architectures. They are about reasoning under uncertainty — and that reasoning is fundamentally mathematical. "How much storage do we need?" "What's our QPS?" "How many servers?" These questions demand quick, principled back-of-envelope calculations. Getting these numbers right (or at least right order-of-magnitude) is what separates a hand-wavy answer from a credible design.

Every section below gives the intuition, the formula, a worked number and where it changes a design decision. The probability behind availability, collisions and tail latency is derived on Probability for System Design; this page applies it.

Back-of-Envelope Estimation Framework

Every estimation follows the same structure:

  1. Define what you are estimating (storage, bandwidth, QPS, latency, cost)
  2. Identify the key variables (users, requests per user, data per request)
  3. Estimate each variable (use round numbers, powers of 10)
  4. Multiply through (chain the estimates)
  5. Sanity check (does the result make sense? Compare to known systems)
graph LR
    A["Define<br/>metric"] --> B["Identify<br/>variables"]
    B --> C["Estimate<br/>each"]
    C --> D["Multiply<br/>through"]
    D --> E["Sanity<br/>check"]

    style A fill:#e3f2fd,color:#000
    style B fill:#e8f5e9,color:#000
    style C fill:#fff3e0,color:#000
    style D fill:#fce4ec,color:#000
    style E fill:#f3e5f5,color:#000

TIP

Interviewers care about your process, not exact numbers. An estimate that is off by 2x but follows sound reasoning is far better than a "correct" number pulled from nowhere. Always show your work.

The chain usually runs in one direction: users drive requests, requests drive QPS, QPS times payload drives bandwidth, writes times retention drives storage, the hot fraction of storage drives memory, and each of those divided by per-machine capacity drives server count and cost.

flowchart LR
    U["DAU and actions per user"] --> Q["QPS: average and peak"]
    Q --> BW["Bandwidth: QPS x payload"]
    Q --> SRV["Servers: peak QPS / per-server QPS"]
    U --> ST["Storage: writes x size x retention x replicas"]
    ST --> MEM["Memory: hot working set"]
    BW --> COST["Cost"]
    ST --> COST
    SRV --> COST

Powers of 2 Table

Memorize this table. It is the foundation of every system design calculation.

PowerExact ValueApproxName
1,0241 Thousand1 KB
1,048,5761 Million1 MB
1,073,741,8241 Billion1 GB
1,099,511,627,7761 Trillion1 TB
~1.13 Quadrillion1 Quadrillion1 PB

Useful Approximations

This means:

  • 1 KB 1,000 bytes
  • 1 MB 1,000 KB bytes
  • 1 GB 1,000 MB bytes
  • 1 TB 1,000 GB bytes

The error compounds: is 10% larger than , which is why a "1 TB" disk shows as about 931 GiB. Fine for estimates; not fine for billing or quota code. Two more facts worth having: (the IPv4 space and the 32-bit integer limit, which an auto-increment ID can overflow), and .

Data Size Reference

Correction (2026-11-09)

"Average 1-min video (720p) ~5 MB" implies about 0.67 Mbps, low for 720p. Typical delivered 720p bitrates are roughly 1–2.5 Mbps depending on codec (H.264 at the top, VP9 and AV1 lower), which is about 7.5–19 MB per minute. The row now says ~10 MB.

Data TypeTypical Size
char / ASCII1 byte
Unicode character (UTF-8)1-4 bytes
32-bit integer4 bytes
64-bit integer / double8 bytes
UUID16 bytes
Timestamp (epoch)8 bytes
MD5 hash16 bytes
SHA-256 hash32 bytes
IPv4 address4 bytes
IPv6 address16 bytes
Average tweet~300 bytes
Average email~50 KB
Average web page~2 MB
Average photo (compressed)~200 KB
Average 1-min video (720p)~10 MB (7.5–19 MB at 1–2.5 Mbps)

Remember per-row overhead: a database row is never just its fields. A PostgreSQL tuple header is 23 bytes plus alignment, each index adds an entry per row, and replication and backups multiply the total. A 100-byte logical record commonly occupies 200–300 bytes on disk once indexes are counted.

Latency Numbers Every Engineer Should Know

Intuition

Different operations differ by eight orders of magnitude. A memory read and a cross-continent round trip differ by a factor of about a million. Design is mostly about avoiding the slow operations on the hot path: keep data in memory, avoid disk seeks, batch network round trips, and never put a cross-region call inside a loop.

The classic list comes from Jeff Dean's talks around 2009–2012. Hardware has moved since; the right-hand column gives rough modern figures. Treat both as orders of magnitude, not benchmarks: they depend on hardware generation, cloud provider, load and configuration.

OperationClassic (~2010)Rough modern figureNotes
L1 cache reference0.5 ns~1 nsA few CPU cycles
Branch mispredict5 ns~3–5 ns
L2 cache reference7 ns~3–5 ns
Mutex lock/unlock (uncontended)25 ns~15–25 nsContended locks are far slower
Main memory reference100 ns~80–100 nsHas barely improved in 15 years
Compress 1 KB (Snappy/LZ4 class)3 µs~1–2 µs
Send 1 KB over the network10 µs at 1 Gbps~1 µs at 10 GbpsSerialization time only, not latency
Random 4 KB read from SSD150 µs~20–100 µs (NVMe)Cloud network block storage: ~0.5–1 ms
Read 1 MB sequentially from memory250 µstens of µs~10–30 GB/s per core
Round trip within a datacenter500 µs~100–500 µsSame zone; cross-zone ~0.5–2 ms
Read 1 MB sequentially from SSD1 ms~0.2–0.3 ms (NVMe)3–7 GB/s
Disk (HDD) seek10 ms~4–10 msUnchanged: it is mechanical
Read 1 MB sequentially from HDD20 ms~5 ms~200 MB/s
Round trip US East ↔ US West~60–70 ms
Round trip California ↔ Netherlands150 ms~140–150 msBounded by physics

The speed of light sets the floor. Light in fibre travels at about m/s, two thirds of its vacuum speed: about 5 µs per km one way, or 1 ms of round trip per 100 km. New York to London is about 5,600 km in a straight line, so at least 56 ms round trip before any routing or processing; real paths are longer. No amount of engineering removes this, which is why global systems replicate data close to users instead of making users call home.

Worked example. A page needs 30 database queries. Inside one zone at ~0.5 ms per round trip, issuing them sequentially costs 15 ms of pure network waiting. If the database is in another region 70 ms away, the same code costs 2.1 seconds. Batch them into one query, or move compute next to the data.

Time Constants

DurationSecondsUseful For
1 minute60Request rates
1 hour3,600Batch processing
1 day86,400 Daily aggregates
1 month2,592,000 Monthly storage
1 year31,536,000 Capacity planning

TIP

The 86,400 trick: A day has 86,400 seconds. Round to for quick math. This means:

  • 1 QPS = ~100K requests/day
  • 1K QPS = ~100M requests/day
  • 10K QPS = ~1B requests/day

Rounding 86,400 up to understates QPS by about 14%. That is fine, but say so, and remember traffic is not spread evenly over 24 hours anyway.

QPS (Queries Per Second) Calculations

From Daily Active Users

Example: Twitter-like service

Peak vs Average

Production systems must handle peak load, not just average. Common rule of thumb:

The multiplier depends on how concentrated usage is. A global consumer app spread across time zones may peak at 2x; a single-country app, a sports app on match day, or a ticketing site at on-sale time can see 10–100x. When you can, take the peak from real data: the busiest minute of the busiest day.

From Concurrent Users

Correction (2026-11-09)

This formula is Little's Law rearranged, and it only holds when "concurrent users" means requests in flight. Real users spend most of their time reading, not waiting on a request. With concurrent users who each issue a request, wait seconds for it and then think for seconds, the throughput is . For 100,000 users with s and s, that is about 3,300 QPS, not the 500,000 the unmodified formula gives.

TypeScript (estimation helper):

interface EstimationParams {
  dau: number;
  requestsPerUserPerDay: number;
  peakMultiplier?: number;
}

function estimateQPS(params: EstimationParams): {
  averageQPS: number;
  peakQPS: number;
  dailyRequests: number;
} {
  const { dau, requestsPerUserPerDay, peakMultiplier = 3 } = params;
  const dailyRequests = dau * requestsPerUserPerDay;
  const averageQPS = dailyRequests / 86_400;
  const peakQPS = averageQPS * peakMultiplier;

  return {
    averageQPS: Math.round(averageQPS),
    peakQPS: Math.round(peakQPS),
    dailyRequests,
  };
}

// Example: Social media
const result = estimateQPS({
  dau: 300_000_000,
  requestsPerUserPerDay: 100,
  peakMultiplier: 3,
});
// { averageQPS: 347222, peakQPS: 1041667, dailyRequests: 30000000000 }

Python:

from dataclasses import dataclass

@dataclass
class QPSEstimate:
    average_qps: int
    peak_qps: int
    daily_requests: int

def estimate_qps(
    dau: int,
    requests_per_user_per_day: int,
    peak_multiplier: float = 3.0
) -> QPSEstimate:
    daily_requests = dau * requests_per_user_per_day
    average_qps = daily_requests / 86_400
    peak_qps = average_qps * peak_multiplier

    return QPSEstimate(
        average_qps=round(average_qps),
        peak_qps=round(peak_qps),
        daily_requests=daily_requests
    )

# Example
result = estimate_qps(dau=300_000_000, requests_per_user_per_day=100)
print(f"Average: {result.average_qps:,} QPS")
print(f"Peak: {result.peak_qps:,} QPS")
# Average: 347,222 QPS
# Peak: 1,041,667 QPS

Storage Calculations

General Formula

Add a factor for indexes and overhead (often 1.3–2x) and for headroom (disks should not run above ~70–80% full; LSM compaction and rebalancing need free space).

Worked Example: URL Shortener

Assumptions:

  • 100M new URLs per day
  • Each URL record: 500 bytes (short URL + long URL + metadata)
  • Retention: 5 years
  • Replication factor: 3

Worked Example: Chat Application

ParameterValue
DAU50M
Messages per user per day40
Average message size200 bytes
Media messages (10% with 200KB avg)200 KB
RetentionForever

Note how media is 99% of the bytes. Text could live in a database; media belongs in object storage behind a CDN, with only a pointer in the message row. Finding the term that dominates is the point of the estimate.

Bandwidth Calculations

Example: If your service handles 10K QPS and each response averages 10 KB:

Bandwidth Conversion

UnitEquivalent
1 Byte8 bits
1 MB/s8 Mbps
1 Gbps125 MB/s
10 Gbps1.25 GB/s

Separate ingress (uploads, writes) from egress (downloads, reads). They have different costs (cloud egress is billed, ingress usually is not) and different bottlenecks. A daily volume converts to an average rate via .

Memory Estimation

Intuition. You rarely cache everything. Access is skewed, so a small hot fraction of the data serves most requests. Memory sizing is: how big is the working set, and what does each entry cost including overhead?

Per-entry overhead is real: a Redis key with a small string value costs on the order of 50–100 bytes beyond the raw bytes (dictionary entry, object headers, allocator rounding). For a million 100-byte values, budget roughly 150–200 MB, not 100 MB.

Worked example. Session store for 100M users, 20% active on a given day, 1 KB per session: GB, plus overhead and a replica, so about 50 GB: a small Redis cluster. Also check connections and per-connection buffers (a WebSocket gateway holding 15 million connections at ~20 KB each needs 300 GB of RAM just for connection state).

Five Complete Worked Estimates

Each estimate lists assumptions first, then derives QPS, storage, bandwidth, memory and machines. The assumptions are illustrative round numbers, not any company's real figures; state yours out loud in an interview.

1. Twitter-Like Social Feed

AssumptionValue
DAU300M
New tweets per day500M
Timeline tweets viewed per user per day100
Tweet size (text + metadata)~1 KB stored, 300 bytes of text
Tweets with an image10%, ~200 KB each
Average followers per tweet author200
Images viewed per user per day20

QPS.

At a 3x peak: about 17,000 writes/s and 1 million timeline-tweet reads/s. The read:write ratio is about 60:1.

Storage. Tweets: GB/day, about 180 TB/year before replication. Images: TB/day, 3.65 PB/year. Media dominates again.

Fan-out-on-write. Writing each tweet ID into every follower's home timeline means timeline inserts per day, about 1.16 million per second on average. At 8 bytes per tweet ID that is 800 GB/day of timeline writes. Capping each cached timeline at 800 entries bounds the memory: TB of RAM, a few dozen large cache nodes. Accounts with millions of followers break fan-out-on-write (one tweet becomes millions of writes), which is why such designs merge celebrity tweets at read time instead.

Bandwidth. Timeline reads: MB/s, about 2.8 Gbps. Images: PB/day, about 111 Gbps on average, which is a CDN problem, not an application-server problem.

Servers. At ~10,000 timeline reads per second per server (served from cache), 1 million peak reads/s needs about 100 servers, plus 2x for redundancy and deploys.

2. URL Shortener, End to End

AssumptionValue
New short URLs per day100M
Read:write ratio100:1
Record size500 bytes
Retention5 years
  • Write QPS: /s; peak about 3,500/s.
  • Read QPS: 100x, about 116,000/s; peak about 350,000/s.
  • ID space: 5 years is URLs. Base-62 with 6 characters gives , not enough; 7 characters give , about 19 times the need. Random 7-character codes need collision checks (the birthday bound bites at about 2.2 million codes; see Probability for System Design).
  • Storage: 273 TB with 3x replication (computed above).
  • Cache: reads concentrate on recent links. Caching the hottest 20% of the last 30 days' links: GB, a handful of cache nodes, which should absorb well over 90% of reads.
  • Bandwidth: MB/s egress, under 0.5 Gbps. Tiny.

The conclusion an interviewer wants: this is a read-heavy, cache-friendly, storage-moderate system. The database sees about 1,200 writes/s and only cache misses for reads; the hard parts are ID generation and redirect latency, not throughput.

3. Chat Application

Same assumptions as the storage example above (50M DAU, 40 messages/user/day, 200-byte text, 10% media at 200 KB), plus: 30% of DAU online at peak, average group size 5.

  • Message QPS: /s; peak about 70,000/s.
  • Deliveries: each message fans out to about 5 recipients in groups, so up to 350,000 deliveries/s at peak, each a push over a persistent connection.
  • Connections: million concurrent WebSocket connections. At ~100,000 connections per gateway (a practical figure, limited by memory and TLS state), that is 150 gateways plus spare. At ~20 KB of state per connection, 300 GB of RAM across them.
  • Storage: 400 GB/day text plus 40 TB/day media (computed above).
  • Bandwidth: text MB/s, negligible; media ingress 40 TB/day MB/s Gbps average, with downloads multiplied by group size.

The bottleneck is connection count and fan-out, not bytes. See the chat schema design page for the data model.

4. Video Upload and Streaming Platform

AssumptionValue
Video uploaded500 hours per minute (the figure YouTube has published since 2019)
Average video length10 minutes
Upload bitrate (1080p source)~5 Mbps
Encoding ladder1080p 5 + 720p 2.5 + 480p 1 + 360p 0.7 + 240p 0.4 = 9.6 Mbps total
Hours watched per day1 billion (YouTube's 2017 figure)
Average delivered bitrate2.5 Mbps
  • Uploads: 500 hours/min = 720,000 hours/day = 4.3 million videos/day = 50 uploads per second. Modest QPS, enormous bytes.
  • Ingest: PB/day of source video.
  • Stored renditions: PB/day, about 9.3 PB/day with 3x replication (or about 4.7 PB/day with erasure coding at 1.5x overhead).
  • Transcoding: if one CPU core needs about 2 hours to transcode one hour of source into the full ladder (an assumption; real figures vary hugely with codec and preset), that is 1.44 million core-hours per day: 60,000 cores running continuously. Codec choice is a compute-versus-bandwidth trade: AV1 saves bytes and costs many times more CPU to encode.
  • Egress: EB/day, about 104 Tbps on average. Only a CDN with servers inside ISPs can carry that; this is the entire business.

5. Metrics and Monitoring System

AssumptionValue
Hosts10,000
Active time series per host1,000
Scrape interval15 s
Raw sample16 bytes (8-byte timestamp + 8-byte float)
Compressed sample~1.37 bytes (Gorilla encoding, Facebook's VLDB 2015 paper)
Raw retention15 days
  • Active series: .
  • Ingest rate: samples/s.
  • Storage: samples/day; raw 920 GB/day, compressed about 79 GB/day. Fifteen days is about 1.2 TB, before replication. Compression is a 12x lever.
  • Memory: the in-memory head block and index cost a few KB per active series in Prometheus-style engines; at ~4 KB that is about 40 GB.
  • Long-term: downsample to 5-minute rollups for a year: points; at a few bytes each, a few TB.

Cardinality is the killer. Add a user_id label with 100,000 values to a metric present on every host and the series count can grow by up to 100,000x; nothing in this estimate survives that. Per-user data belongs in logs or traces, not metric labels. See Time-Series Databases.

Little's Law and Utilization

Little's Law

Intuition. In any stable system, the average number of things inside equals the rate they arrive times how long each stays. A shop with 10 customers arriving per minute who each stay 3 minutes has 30 customers inside on average.

It holds for any arrival pattern and any service-time distribution, as long as the system is stable (arrivals do not outpace departures forever).

Worked examples.

  • Concurrency: 2,000 requests/s at 150 ms average latency means requests in flight. With 50 worker threads per instance, you need at least 6 instances, before headroom.
  • Connection pools: 5,000 queries/s at 4 ms each keeps connections busy on average. A pool of 20 is exactly saturated and will queue constantly; size for the peak and for latency spikes (if query latency rises to 40 ms during a slow period, the same load needs 200 connections). See Connection Pooling.
  • Queues: a Kafka consumer group with 30,000 messages of lag draining at 1,000 messages/s has 30 seconds of delay.

Utilization vs Latency

Intuition. A server that is busy 90% of the time is idle only 10% of the time, so a new request is likely to find a queue. As utilization approaches 1, queues grow without bound. For the simplest model (M/M/1: random arrivals, random service times, one server), the average time in system is

where is the service time. With ms:

Utilization Average latency
50%20 ms
70%33 ms
80%50 ms
90%100 ms
95%200 ms
99%1,000 ms

Going from 80% to 90% busy doubles latency; 90% to 99% multiplies it by ten. This curve is why services target 50–70% utilization at peak, why "the CPU is only at 85%" can coexist with terrible latency, and why a small traffic increase near saturation causes a large latency jump. Multi-server pools (M/M/c) behave better at the same utilization, which is one argument for fewer, larger pools over many small ones. The full treatment is on Queueing Theory.

def mm1_latency(service_ms: float, utilization: float) -> float:
    """Average time in an M/M/1 system."""
    if utilization >= 1:
        return float("inf")
    return service_ms / (1 - utilization)

def instances_needed(peak_rps: float, service_ms: float, workers_per_instance: int,
                     target_util: float = 0.6) -> int:
    """Little's Law concurrency, then divide by capacity at the target utilization."""
    in_flight = peak_rps * service_ms / 1000
    return int(-(-in_flight // (workers_per_instance * target_util)))  # ceiling

for rho in (0.5, 0.8, 0.9, 0.99):
    print(f"utilization {rho:.0%}: {mm1_latency(10, rho):7.1f} ms")
print("instances for 6,000 rps at 150 ms, 50 workers each, 60% target:",
      instances_needed(6_000, 150, 50))
# utilization 50%:    20.0 ms
# utilization 80%:    50.0 ms
# utilization 90%:   100.0 ms
# utilization 99%:  1000.0 ms
# instances for 6,000 rps at 150 ms, 50 workers each, 60% target: 30

Availability Math and Nines Budgets

Composing Availability

A request path that needs every component is a series system; availability multiplies. Redundant copies form a parallel system; unavailability multiplies.

Worked example. A service promising 99.99% calls 10 dependencies serially, each also 99.99%: . It cannot meet its own SLO, even with perfect code. Either the dependencies must be better than the service's own target, or calls must be made non-critical (cached fallbacks, timeouts with degraded responses, asynchronous work).

Nines as Budgets

AvailabilityDowntime per yearPer 30-day monthError budget at 1B requests/month
99%3.65 days7.2 hours10M failed requests
99.9%8.77 hours43.2 minutes1M
99.95%4.38 hours21.6 minutes500K
99.99%52.6 minutes4.3 minutes100K
99.999%5.26 minutes26 seconds10K

Budgets drive policy. At 99.95% you have 21.6 minutes a month. If a bad deploy typically costs 10 minutes before rollback, you can afford two bad deploys a month, which tells you how much to invest in canarying and automatic rollback. At 99.99% (4.3 minutes) no human-in-the-loop recovery fits; detection and failover must be automatic.

Two more rules of thumb. Availability from MTBF and MTTR is , so halving repair time is as good as doubling time between failures. And all the parallel math assumes independent failures; correlated failures (a shared zone, a shared config push) usually dominate real outages. Both are derived on Probability for System Design.

Replication Factor Calculations

Write Amplification

With a replication factor of :

In other words, in single-leader replication the leader handles every write, so adding followers scales reads but not writes; to scale writes you shard. Storage engines add their own amplification on top: an LSM tree may rewrite each byte 10–30 times through compaction, so 100 MB/s of logical writes at RF 3 can mean several GB/s of physical disk writes across the cluster.

Quorum Reads and Writes

For a system with replicas, a write quorum , and a read quorum :

Common configurations:

ConfigTrade-off
Strong consistency322Balanced
Write-optimized313Fast writes, slow reads
Read-optimized331Slow writes, fast reads
Eventual consistency311Fast but stale reads possible

The overlap condition is pigeonhole arithmetic: if , any read set and any write set share at least one replica, so a read sees the latest acknowledged write. (Strictly, "strong consistency" also needs correct handling of concurrent writes, sloppy quorums and failed writes; Quorums and Leaderless Replication covers the edge cases.)

Quorum latency is an order statistic. A quorum operation finishes when the -th (or -th) fastest replica answers. With and , you wait for the second-fastest of three, which is faster than waiting for a single specific replica's tail, because one slow replica is simply ignored. With you wait for the slowest. That is why hurts tail latency and availability at once.

Quorum availability is a binomial tail. With 3 replicas each up 99.9% of the time, succeeds with probability ; only .

Durability: Why Repair Time Matters

Intuition. With 3 replicas, data is lost only if all three copies die before the system re-replicates. The danger window is the repair time .

For a disk annual failure rate (on the order of 1–2% for modern drives in large fleets, per Backblaze's published drive statistics) and hourly failure rate , a rough estimate per chunk per year is

Worked example. AFR 2%, hour: per chunk-year. With chunks, about expected loss events per year. With hours: per chunk, or 0.36 loss events per year, almost 600 times worse, because the risk scales with . Fast, parallel re-replication (many nodes each copying a little) matters more than a fourth replica. Correlated failures (a power event taking out a rack) make all of this worse; that is what rack- and zone-aware placement is for.

Availability Calculation

For independent replicas, each with availability :

Single node availability2 replicas3 replicas5 replicas
99% (3.65 days downtime)99.99%99.9999%99.99999999%
99.9% (8.77 hrs downtime)99.9999%~100%~100%
99.99% (52.6 min downtime)~100%~100%~100%

WARNING

These calculations assume independent failures. In practice, correlated failures (data center outages, shared dependencies) make real availability lower. This is why multi-region deployment matters.

This table is for "any one replica can serve" (reads from any copy). For majority quorums, use the binomial tail above instead: 3 nodes at 99% give 99.97%, not 99.9999%.

Consistent Hashing Math

Consistent hashing maps both keys and servers onto a ring of size (typically or for MD5/SHA-1). Each key is assigned to the next server clockwise on the ring.

graph TD
    subgraph "Hash Ring"
        direction TB
        S1["Server A<br/>(position 30)"]
        S2["Server B<br/>(position 120)"]
        S3["Server C<br/>(position 220)"]
        K1["Key 1<br/>(hash: 50) → B"]
        K2["Key 2<br/>(hash: 150) → C"]
        K3["Key 3<br/>(hash: 250) → A"]
    end

Virtual Nodes

With physical servers, each server gets virtual nodes on the ring. This improves load distribution.

With virtual nodes per server, the standard deviation of load drops to about 5-10% of the mean — acceptable for most systems.

Why . Each server's share of the ring is the sum of arc lengths. With one point per server, arcs are exponentially distributed and wildly uneven: the standard deviation of a server's share is about equal to its mean. Summing independent arcs shrinks the relative spread by , the same law as in sampling. A simulation with 10 servers (MD5 positions) confirms it:

Virtual nodes Std dev of load (fraction of mean)Busiest serverLightest server
194%3.26x mean0.12x
1029%1.29x0.52x
1007.2%1.12x0.87x
1,0003.0%1.04x0.93x
import hashlib
import numpy as np

def ring_load_shares(n_servers: int, vnodes: int) -> np.ndarray:
    """Fraction of the hash ring owned by each server (1.0 = fair share)."""
    points = sorted(
        (int.from_bytes(hashlib.md5(f"node{s}:{i}".encode()).digest()[:8], "big"), s)
        for s in range(n_servers) for i in range(vnodes)
    )
    share = np.zeros(n_servers)
    ring = 2**64
    for idx, (pos, server) in enumerate(points):
        prev = points[idx - 1][0] - (ring if idx == 0 else 0)
        share[server] += (pos - prev) / ring   # arc from the previous point is owned by this one
    return share * n_servers

for v in (1, 10, 100, 1000):
    s = ring_load_shares(10, v)
    print(f"V={v:>4}: std={s.std():.3f}  max={s.max():.2f}  min={s.min():.2f}")
# V=   1: std=0.942  max=3.26  min=0.12
# V=  10: std=0.285  max=1.29  min=0.52
# V= 100: std=0.072  max=1.12  min=0.87
# V=1000: std=0.030  max=1.04  min=0.93

The busiest server, not the average, sets capacity: at you must provision every server for 1.29x the mean, wasting 29% of the fleet. Virtual nodes cost memory for the ring and make membership changes touch more ranges, so 100–256 per server is a common compromise.

Rebalancing on Server Addition/Removal

When a server is added, only keys need to be remapped (where is total keys, is total servers). Compare this to modular hashing where nearly all keys must move:

Correction (2026-11-09)

Strictly, adding a server to existing ones moves about keys (the new server's fair share), as the table below says; is the figure for removing one of servers. For large the two are nearly equal.

EventConsistent HashingModular Hashing
Add 1 server ( to ) keys move keys move
Remove 1 server keys move keys move

Worked example. 10 servers holding 1 billion keys. Adding an 11th: consistent hashing moves about 91 million keys; modular hashing (hash % N) moves about 909 million, and every one of those is a cache miss or a data copy at the same moment.

TypeScript:

import { createHash } from "crypto";

class ConsistentHash {
  private ring: Map<number, string> = new Map();
  private sortedKeys: number[] = [];
  private virtualNodes: number;

  constructor(virtualNodes = 150) {
    this.virtualNodes = virtualNodes;
  }

  private hash(key: string): number {
    const h = createHash("md5").update(key).digest();
    return h.readUInt32BE(0);
  }

  addServer(server: string): void {
    for (let i = 0; i < this.virtualNodes; i++) {
      const virtualKey = this.hash(`${server}:${i}`);
      this.ring.set(virtualKey, server);
      this.sortedKeys.push(virtualKey);
    }
    this.sortedKeys.sort((a, b) => a - b);
  }

  removeServer(server: string): void {
    for (let i = 0; i < this.virtualNodes; i++) {
      const virtualKey = this.hash(`${server}:${i}`);
      this.ring.delete(virtualKey);
    }
    this.sortedKeys = this.sortedKeys.filter((k) => this.ring.has(k));
  }

  getServer(key: string): string | undefined {
    if (this.sortedKeys.length === 0) return undefined;

    const h = this.hash(key);
    // Binary search for next server clockwise
    let lo = 0, hi = this.sortedKeys.length;
    while (lo < hi) {
      const mid = (lo + hi) >> 1;
      if (this.sortedKeys[mid] < h) lo = mid + 1;
      else hi = mid;
    }

    const idx = lo % this.sortedKeys.length;
    return this.ring.get(this.sortedKeys[idx]);
  }
}

Python:

import hashlib
from bisect import bisect_right

class ConsistentHash:
    def __init__(self, virtual_nodes: int = 150):
        self.virtual_nodes = virtual_nodes
        self.ring: dict[int, str] = {}
        self.sorted_keys: list[int] = []

    def _hash(self, key: str) -> int:
        h = hashlib.md5(key.encode()).digest()
        return int.from_bytes(h[:4], "big")

    def add_server(self, server: str) -> None:
        for i in range(self.virtual_nodes):
            virtual_key = self._hash(f"{server}:{i}")
            self.ring[virtual_key] = server
        self.sorted_keys = sorted(self.ring.keys())

    def remove_server(self, server: str) -> None:
        for i in range(self.virtual_nodes):
            virtual_key = self._hash(f"{server}:{i}")
            self.ring.pop(virtual_key, None)
        self.sorted_keys = sorted(self.ring.keys())

    def get_server(self, key: str) -> str | None:
        if not self.sorted_keys:
            return None

        h = self._hash(key)
        idx = bisect_right(self.sorted_keys, h)
        if idx == len(self.sorted_keys):
            idx = 0
        return self.ring[self.sorted_keys[idx]]

# Check the rebalancing claim: adding an 11th server should move about 1/11 of keys
ch = ConsistentHash()
for s in range(10):
    ch.add_server(f"server-{s}")
keys = [f"user:{i}" for i in range(100_000)]
before = {k: ch.get_server(k) for k in keys}
ch.add_server("server-10")
moved = sum(before[k] != ch.get_server(k) for k in keys)
print(f"moved {moved / len(keys):.1%} of keys (ideal 1/11 = {1 / 11:.1%})")
# moved 9.4% of keys (ideal 1/11 = 9.1%)

Sharding Math: Hot Partitions

Intuition

Hashing spreads keys evenly. It does nothing for traffic when a few keys are far more popular than the rest. Real popularity is often Zipf-like: the -th most popular key gets traffic proportional to with near 1.

With keys and , the most popular key's share of all traffic is

Worked Example

One million keys, Zipf with : , so the top key gets 6.9% of all traffic, the top 10 keys 20%, and the top 1,000 keys 52%. Spread across 100 shards (average 1% each), the shard holding the top key carries about 7.9% of traffic: 7.8 times the average (simulated with random key placement). Adding shards does not help: the top key's 6.9% is on one shard no matter how many there are.

import numpy as np

rng = np.random.default_rng(0)
n_keys, n_shards = 1_000_000, 100
weights = 1 / np.arange(1, n_keys + 1)            # Zipf, s = 1
weights /= weights.sum()
shard_of_key = rng.integers(0, n_shards, n_keys)  # hash placement
load = np.bincount(shard_of_key, weights=weights, minlength=n_shards)
print(f"top key share: {weights[0]:.1%}")
print(f"hottest shard: {load.max() / load.mean():.1f}x average, coolest: {load.min() / load.mean():.2f}x")
# top key share: 6.9%
# hottest shard: 7.8x average, coolest: 0.50x

Remedies. Cache the hot keys in front of the shards (the cache absorbs exactly the skew that hurts); replicate hot keys across several shards for reads; split hot keys for writes by appending a suffix (counter:42#0 … counter:42#9) and summing on read; and detect hot keys from live traffic sampling rather than guessing. DynamoDB's adaptive capacity and "split for heat" exist because of this arithmetic. See Sharding.

Shard count rule of thumb. Size so that the hottest shard at peak stays under its limit, not the average one, and keep enough shards that one shard's failure or rebalancing touches a small fraction of data. Many systems pre-split into many more logical shards (hundreds or thousands) than physical nodes so that rebalancing moves whole logical shards.

Cache Hit Ratio and Backend Load

Intuition

The backend only sees misses. So the number that matters is not the hit ratio but the miss ratio, and small changes in hit ratio near 100% are huge changes in miss ratio.

The 90% → 99% Insight

Hit ratio Backend load (of total)Average latency (1 ms hit, 50 ms miss)
90%10%5.9 ms
95%5%3.5 ms
99%1%1.5 ms
99.9%0.1%1.05 ms

Going from 90% to 99% sounds like a 10% improvement. It is a 10x reduction in database load and a 4x reduction in average latency. Conversely, a cache that drops from 99% to 90% (a bad deploy that changes key formats, an eviction storm, a cold restart) multiplies backend load by 10 instantly. If the database was sized for the 99% world, it falls over.

Worked example. 100,000 reads/s with a 99% hit ratio: the database serves 1,000 reads/s, comfortable for one primary and a replica. Restart the cache fleet and the hit ratio briefly goes to 0%: the database receives 100,000 reads/s, 100 times its normal load. This is why cache restarts are rolled, why caches are warmed before taking traffic, and why request coalescing exists. See Cache Sizing Math and Thundering Herd.

def backend_qps(total_qps: float, hit_ratio: float) -> float:
    return total_qps * (1 - hit_ratio)

for h in (0.90, 0.99, 0.999):
    print(f"hit {h:.1%}: backend sees {backend_qps(100_000, h):>8,.0f} QPS")
print(f"cache cold (0%): backend sees {backend_qps(100_000, 0):,.0f} QPS")
# hit 90.0%: backend sees   10,000 QPS
# hit 99.0%: backend sees    1,000 QPS
# hit 99.9%: backend sees      100 QPS
# cache cold (0%): backend sees 100,000 QPS

Probabilistic Data-Structure Sizing

When exact answers would need memory proportional to the data, a small, tunable error buys enormous savings.

Bloom Filter

Intuition. A bit array plus hash functions answers "definitely not present" or "probably present". It never gives false negatives; the false-positive rate is set by bits per element.

Worked example. A crawler tracking 1 billion seen URLs at a 1% false-positive rate needs bits per URL: about 1.2 GB with . Storing the URLs themselves (about 70 bytes average) would take roughly 70 GB. Each tenfold reduction in costs another 4.8 bits per element. A false positive here means skipping a URL that was never crawled, which is acceptable; for "has this password been breached", a false positive needs a confirming lookup. Derivation and variants: Bloom Filters.

HyperLogLog

Intuition. To count distinct items, hash each one and watch for unusually long runs of leading zeros: seeing a hash starting with 20 zeros suggests you have seen around distinct items. Averaging this over independent buckets (registers) tames the noise.

Worked example. Redis's HyperLogLog uses registers of 6 bits: 12 KB per counter, with standard error , whether it counts a thousand or a billion items. Counting daily unique visitors for 10 million pages exactly would need a set per page; with HLL it is at most 120 GB (less with Redis's sparse encoding for small counts), and HLLs merge, so weekly uniques are the union of seven daily sketches without rescanning.

Count-Min Sketch

For "how often has this item appeared" (heavy hitters, hot-key detection), a count-min sketch with width and depth overestimates a count by at most with probability . With and : , , about 13,600 counters, or 54 KB at 4 bytes each, regardless of how many distinct items flow through.

StructureAnswersMemory exampleError
Bloom filterIs in the set?1.2 GB for 1B items1% false positives, no false negatives
HyperLogLogHow many distinct?12 KB per counter~0.81% standard error
Count-min sketchHow often did appear?54 KBOverestimates by at most 0.1% of the total
import math

def bloom_size(n: int, p: float) -> tuple[float, int]:
    bits = -n * math.log(p) / math.log(2) ** 2
    return bits / 8 / 1e9, round(bits / n * math.log(2))

gb, k = bloom_size(1_000_000_000, 0.01)
print(f"Bloom filter, 1B items at 1%: {gb:.2f} GB, k={k}")
print(f"HyperLogLog, 16384 registers: {16384 * 6 // 8:,} bytes, error {1.04 / math.sqrt(16384):.2%}")
w, d = math.ceil(math.e / 0.001), math.ceil(math.log(1 / 0.01))
print(f"Count-min sketch eps=0.001 delta=0.01: {w} x {d} = {w * d * 4 / 1024:.0f} KB")
# Bloom filter, 1B items at 1%: 1.20 GB, k=7
# HyperLogLog, 16384 registers: 12,288 bytes, error 0.81%
# Count-min sketch eps=0.001 delta=0.01: 2719 x 5 = 53 KB

Tail-Latency Amplification

Intuition. A request that waits on many backends is as slow as the slowest. If each backend is slow with probability (say for "slower than its p99"), a request touching of them in parallel is slow with probability

Worked example. Fan-out of 100: of requests hit at least one backend's p99. The backend's p99 has become the user's median. To keep the user-facing p99 at the backend's p99 with fan-out 100, each backend needs that latency at its p99.99. The effect also applies to sequential calls: 5 sequential services each with a 1% slow tail give slow requests.

Design consequences. Budget latency per hop (if the user SLO is 200 ms at p99 and the request makes 4 sequential calls, each gets roughly a 50 ms p99.9 target, not 50 ms p99); use hedged requests and partial results for wide fan-out; keep fan-out small on latency-critical paths. The derivation and a simulation are on Probability for System Design.

Cost Estimation

Intuition

Cost is the estimate that reaches finance, and it is usually dominated by one term. Multiply each resource by its unit price, find the biggest line, and design around it.

Approximate unit prices (AWS us-east-1 list prices as of 2025; they change, vary by region and commitment, and every provider differs, so check current pricing before relying on them):

ResourceApproximate list price
Object storage (S3 Standard, first 50 TB)USD 0.023 per GB-month
Block storage (EBS gp3)USD 0.08 per GB-month
Internet egress (first 10 TB/month, falling with volume)USD 0.09 per GB, about USD 0.05 per GB above 150 TB
General-purpose VM, 2 vCPU / 8 GB (m7i.large, on-demand)about USD 0.10 per hour, USD 74 per month

Worked Examples

  • URL shortener storage: 273 TB on block storage at USD 0.08/GB-month is about USD 22,000 a month; as object storage it would be USD 6,300, but object storage is too slow for redirect lookups. The real choice is a database on SSD, so use the block-storage figure, and notice it is larger than the compute bill.
  • Chat media: 40 TB/day is 14.6 PB/year. At USD 0.023/GB-month, a year of media costs about USD 336,000 per month to store by the end of the year, and the bill grows every month because retention is "forever". Infrequent-access and archive tiers (several times cheaper) for old media are the obvious lever.
  • Egress: serving 1 PB a month from a cloud region at about USD 0.05/GB is about USD 50,000 a month, before CDN discounts. Egress is frequently the largest line for media products, and why CDNs and peering agreements matter.
  • Compute: 100 servers of the m7i.large class is about USD 7,400 a month on demand; 1- to 3-year commitments typically cut that by roughly 30–60%.

The pattern: storage grows forever, egress scales with popularity, compute scales with peak. A cost estimate that ignores growth (storage at month 1 rather than month 36) is off by an order of magnitude. See FinOps cost optimization.

CAP Theorem: Formal Intuition

The CAP theorem states that a distributed system can provide at most two of three guarantees simultaneously:

  • Consistency (C): Every read receives the most recent write
  • Availability (A): Every request receives a response (not an error)
  • Partition tolerance (P): The system continues to operate despite network partitions

Why You Must Choose

During a network partition, a message from node A cannot reach node B. You have two options:

  1. Choose Consistency (CP): Refuse to respond until the partition heals. You lose availability.
  2. Choose Availability (AP): Respond with potentially stale data. You lose consistency.

Since network partitions are inevitable in distributed systems, the real choice is between CP and AP during partition events.

graph TD
    P["Network Partition<br/>Occurs"] --> Q{"Respond to<br/>client?"}
    Q -->|"Yes, even with<br/>stale data"| AP["AP System<br/>(Cassandra, DynamoDB)"]
    Q -->|"No, wait for<br/>consensus"| CP["CP System<br/>(HBase, MongoDB)"]

    style AP fill:#ff9800,color:#fff
    style CP fill:#2196f3,color:#fff

PACELC Extension

PACELC extends CAP: if there is a Partition, choose A or C; Else (normal operation), choose Latency or Consistency.

SystemDuring PartitionNormal Operation
DynamoDBAPEL (low latency)
MongoDBCPEC (consistent)
CassandraAPEL (tunable)
PostgreSQLCPEC (ACID)

Note (2026-11-09)

These labels depend on configuration. Abadi's original PACELC paper (IEEE Computer, 2012) classified MongoDB as PA/EC; with majority write concern and primary reads it behaves as PC/EC, as the table shows. DynamoDB with strongly consistent reads and Cassandra at QUORUM move toward C. Classify the configuration, not the product. See CAP Theorem.

Estimation Cheat Sheet

Common Service Numbers

Note (2026-11-09)

These rows are illustrative orders of magnitude under assumptions like those in the worked estimates above, not published figures. By this page's own Twitter-like estimate, ~300K is the average read QPS; the peak would be closer to 1M.

ServiceDAUPeak QPSStorage/day
Twitter-scale300M~300K~15 TB
Instagram-scale500M~500K~100 TB (images)
WhatsApp-scale2B~1M~50 TB
YouTube-scale2B~200K~500 TB (video)
URL shortener100M~10K~50 GB

Server Capacity Rules of Thumb

ResourceSingle ServerNotes
QPS (web server)1,000-10,000Depends on response time
QPS (database)1,000-5,000With indexing
QPS (cache - Redis)100,000+In-memory
RAM64-256 GBCommodity
Disk1-10 TB SSDFast I/O
Network1-10 GbpsStandard NIC

The database row is deliberately conservative: a well-indexed PostgreSQL or MySQL server on modern NVMe can serve tens of thousands of simple point reads per second, but complex queries, writes with many indexes, and lock contention pull real numbers far lower. Measure your own workload. Cloud instances commonly offer 10–25 Gbps or more today.

Quick Estimation Formulas

The Estimation Cheat Table

QuantityFormulaHandy number
Average QPSdaily requests / 86,4001M/day ≈ 12/s; 1B/day ≈ 11,600/s
Peak QPS2–5 x average (more for spiky products)
BandwidthQPS x payload1 TB/day ≈ 93 Mbps average
Storagewrites/day x size x days x replicas x overhead1 KB x 1B/day ≈ 1 TB/day
ConcurrencyLittle: 1,000/s x 200 ms = 200 in flight
Queue latency for M/M/190% busy = 10x service time
Series availability10 deps at 99.99% = 99.9%
Parallel availability2 x 99.9% = 99.9999% if independent
Backend load x QPS90% → 99% hits = 10x less load
Fan-out tailn = 100, p = 1% gives 63%
Bloom filter1.44 log2(1/p) bits per item1% ≈ 9.6 bits
HyperLogLogerror ≈ 1.04/√m12 KB ≈ 0.81%
Speed of light in fibre~200 km per ms1 ms RTT per 100 km

Common Estimation Mistakes

  1. Using average instead of peak. Capacity is set by the busiest minute. Average QPS times a stated peak factor, or the measured peak.
  2. Forgetting replication, indexes and headroom. Logical bytes x 3 replicas x 1.5 overhead / 0.7 fill factor is about 6.4 times the naive number.
  3. Mixing bits and bytes. Network is quoted in bits per second, storage in bytes. 1 Gbps is 125 MB/s, not 1 GB/s.
  4. Confusing GB and GiB at scale. 10% at the TB level, 13% at the PB level.
  5. Assuming uniform load. Hot keys, celebrity accounts and diurnal peaks break per-shard averages. Size the hottest shard.
  6. Assuming independent failures. Three replicas in one zone are one replica with extra steps.
  7. Running servers near 100% utilization. Latency is ; plan for 50–70% at peak.
  8. Reading "QPS = users / latency" as user count. It is in-flight requests; real users mostly think.
  9. Ignoring growth. Storage at month 36, not month 1. Retention "forever" means cost grows forever.
  10. Precision theatre. "347,222 QPS" signals false accuracy; "about 350K, peak about 1M" is the right register.

Practice Estimation Problems

Correction (2026-11-09)

The YouTube row's variables did not produce its answer: 500 hours per minute is 720,000 hours (2.6 billion seconds) per day, and at 5 MB/s that is about 13 PB/day, not 2.5 PB. About 2.5 PB/day corresponds to roughly 1 MB/s (8 Mbps) per stored hour across all renditions; the row now shows both. The fan-out row now states its assumptions (200M tweets/day, ~30 bytes per timeline entry) so that "~3 TB" can be reproduced: .

ProblemKey VariablesExpected Answer
"Estimate YouTube storage per day"500 hrs video/min, 5 MB/sec, resolutions~13 PB/day at 5 MB/s (~2.5 PB/day at ~1 MB/s)
"How many servers for 1M QPS?"5K QPS/server~200 servers
"Twitter fan-out storage"300M users, 500 avg followers, 200M tweets/day, ~30 B per timeline entry~3 TB write amplification/day
"Cache size for 100M users"20% active, 1 KB per session~20 GB

TIP

When in doubt during an interview, round aggressively and state your assumptions clearly. " seconds in a day" is close enough and much easier to multiply than 86,400. Interviewers value clarity of reasoning over arithmetic precision.

Interview Questions

1. Estimate the QPS, storage and cache size for a URL shortener with 100M new links a day.

Answer. Writes /s (peak about 3,500); reads at 100:1, about 116,000/s (peak about 350,000). Storage at 500 bytes for 5 years: 91 TB, 273 TB with RF 3. 7-character base-62 codes give IDs, enough for the needed; random codes need collision checks. Cache the hot fraction of recent links, about 300 GB. Bandwidth under 0.5 Gbps. The system is read-heavy and cache-friendly; the database mostly sees writes and misses.

2. Your cache hit ratio improves from 90% to 99%. How much does database load change?

Answer. The database sees misses: 10% of traffic becomes 1%, a 10x reduction. Average latency with 1 ms hits and 50 ms misses goes from 5.9 ms to 1.5 ms. The flip side: a cold cache or key-format change that drops the hit ratio to 90% multiplies DB load by 10, so size and protect the DB (rolling restarts, warming, request coalescing) for that case.

3. A service handles 2,000 RPS at 150 ms average latency. How many concurrent requests, and what happens if latency doubles?

Answer. Little's Law: in flight. If a dependency slows and latency doubles to 300 ms at the same arrival rate, concurrency doubles to 600. If the thread or connection pool was sized at 400, requests now queue, latency grows further, and the system can spiral. Size pools for peak latency, and use timeouts and load shedding to cap concurrency.

4. Why do services target 60% utilization rather than 90%?

Answer. Queueing delay grows like . For M/M/1 with 10 ms service time, latency is 25 ms at 60%, 100 ms at 90% and 1 s at 99%. At 90% a small traffic spike or a slow host pushes the system into the steep part of the curve. Headroom also covers losing a zone (with 3 zones, surviving one loss needs each to run below about 67%).

5. A product promises 99.99% availability and calls 10 services in series, each at 99.99%. Is that possible?

Answer. No: . Either the dependencies must be much better (around 99.999% each), or the calls must be made non-critical: cache fallbacks, degraded responses, asynchronous processing, or parallel redundant paths. At 99.99% the monthly budget is 4.3 minutes, so recovery must also be automatic.

6. With 1 million keys following a Zipf distribution, will adding shards fix a hot shard?

Answer. No. The top key alone has about of traffic, all on one shard; with 100 shards that shard runs at about 7.8x average no matter how many shards exist. Fixes target the key: a cache tier in front, read replicas of hot keys, splitting hot write keys into sub-keys and summing on read, and hot-key detection from sampled traffic.

7. How much memory does a Bloom filter need for 1 billion URLs at a 1% false-positive rate? And to count unique visitors?

Answer. bits per item: about 1.2 GB with 7 hash functions, versus about 70 GB to store the URLs. For unique visitors, a HyperLogLog: 12 KB per counter at about 0.81% standard error, mergeable across days.

8. Your search aggregator fans out to 100 shards. Each shard's p99 is 40 ms. What is the user's p99?

Answer. of requests wait on at least one shard's p99, so 40 ms is roughly the user's median; the user p99 is set by the shards' p99.99. Mitigations: hedged requests after the shard p95, returning partial results, fewer and larger shards on the hot path, and fixing the shard's tail causes.

9. Data is stored with 3 replicas. Is a fourth replica or faster re-replication the better durability investment?

Answer. Loss requires all replicas to fail within the repair window , and the probability scales with for 3 replicas. Cutting repair time from 24 hours to 1 hour reduces loss probability about 576-fold (in the worked example, from 0.36 to 0.0006 expected loss events per year across a billion chunks). Parallel re-replication is usually cheaper than 33% more storage, though a fourth replica in another zone also guards against correlated failures, which this model ignores.

10. Estimate the monthly egress bill for a video app serving 1 PB a month, and what you would do about it.

Answer. At roughly USD 0.05 per GB at high-volume tiers, 1 PB ( GB) is about USD 50,000 a month (more at lower tiers). Levers: a CDN with negotiated rates or ISP-embedded caches, better codecs (fewer bytes per hour of video, traded against encoding CPU), adaptive bitrate so mobile users do not fetch 1080p, and caching popular content at the edge.

Further Reading