Math Patterns in System Design
System design interviews are not about memorizing architectures. They are about reasoning under uncertainty — and that reasoning is fundamentally mathematical. "How much storage do we need?" "What's our QPS?" "How many servers?" These questions demand quick, principled back-of-envelope calculations. Getting these numbers right (or at least right order-of-magnitude) is what separates a hand-wavy answer from a credible design.
Every section below gives the intuition, the formula, a worked number and where it changes a design decision. The probability behind availability, collisions and tail latency is derived on Probability for System Design; this page applies it.
Back-of-Envelope Estimation Framework
Every estimation follows the same structure:
- Define what you are estimating (storage, bandwidth, QPS, latency, cost)
- Identify the key variables (users, requests per user, data per request)
- Estimate each variable (use round numbers, powers of 10)
- Multiply through (chain the estimates)
- Sanity check (does the result make sense? Compare to known systems)
graph LR
A["Define<br/>metric"] --> B["Identify<br/>variables"]
B --> C["Estimate<br/>each"]
C --> D["Multiply<br/>through"]
D --> E["Sanity<br/>check"]
style A fill:#e3f2fd,color:#000
style B fill:#e8f5e9,color:#000
style C fill:#fff3e0,color:#000
style D fill:#fce4ec,color:#000
style E fill:#f3e5f5,color:#000
TIP
Interviewers care about your process, not exact numbers. An estimate that is off by 2x but follows sound reasoning is far better than a "correct" number pulled from nowhere. Always show your work.
The chain usually runs in one direction: users drive requests, requests drive QPS, QPS times payload drives bandwidth, writes times retention drives storage, the hot fraction of storage drives memory, and each of those divided by per-machine capacity drives server count and cost.
flowchart LR
U["DAU and actions per user"] --> Q["QPS: average and peak"]
Q --> BW["Bandwidth: QPS x payload"]
Q --> SRV["Servers: peak QPS / per-server QPS"]
U --> ST["Storage: writes x size x retention x replicas"]
ST --> MEM["Memory: hot working set"]
BW --> COST["Cost"]
ST --> COST
SRV --> COST
Powers of 2 Table
Memorize this table. It is the foundation of every system design calculation.
| Power | Exact Value | Approx | Name |
|---|---|---|---|
| 1,024 | 1 Thousand | 1 KB | |
| 1,048,576 | 1 Million | 1 MB | |
| 1,073,741,824 | 1 Billion | 1 GB | |
| 1,099,511,627,776 | 1 Trillion | 1 TB | |
| ~1.13 Quadrillion | 1 Quadrillion | 1 PB |
Useful Approximations
This means:
- 1 KB 1,000 bytes
- 1 MB 1,000 KB bytes
- 1 GB 1,000 MB bytes
- 1 TB 1,000 GB bytes
The error compounds: is 10% larger than , which is why a "1 TB" disk shows as about 931 GiB. Fine for estimates; not fine for billing or quota code. Two more facts worth having: (the IPv4 space and the 32-bit integer limit, which an auto-increment ID can overflow), and .
Data Size Reference
Correction (2026-11-09)
"Average 1-min video (720p) ~5 MB" implies about 0.67 Mbps, low for 720p. Typical delivered 720p bitrates are roughly 1–2.5 Mbps depending on codec (H.264 at the top, VP9 and AV1 lower), which is about 7.5–19 MB per minute. The row now says ~10 MB.
| Data Type | Typical Size |
|---|---|
| char / ASCII | 1 byte |
| Unicode character (UTF-8) | 1-4 bytes |
| 32-bit integer | 4 bytes |
| 64-bit integer / double | 8 bytes |
| UUID | 16 bytes |
| Timestamp (epoch) | 8 bytes |
| MD5 hash | 16 bytes |
| SHA-256 hash | 32 bytes |
| IPv4 address | 4 bytes |
| IPv6 address | 16 bytes |
| Average tweet | ~300 bytes |
| Average email | ~50 KB |
| Average web page | ~2 MB |
| Average photo (compressed) | ~200 KB |
| Average 1-min video (720p) | ~10 MB (7.5–19 MB at 1–2.5 Mbps) |
Remember per-row overhead: a database row is never just its fields. A PostgreSQL tuple header is 23 bytes plus alignment, each index adds an entry per row, and replication and backups multiply the total. A 100-byte logical record commonly occupies 200–300 bytes on disk once indexes are counted.
Latency Numbers Every Engineer Should Know
Intuition
Different operations differ by eight orders of magnitude. A memory read and a cross-continent round trip differ by a factor of about a million. Design is mostly about avoiding the slow operations on the hot path: keep data in memory, avoid disk seeks, batch network round trips, and never put a cross-region call inside a loop.
The classic list comes from Jeff Dean's talks around 2009–2012. Hardware has moved since; the right-hand column gives rough modern figures. Treat both as orders of magnitude, not benchmarks: they depend on hardware generation, cloud provider, load and configuration.
| Operation | Classic (~2010) | Rough modern figure | Notes |
|---|---|---|---|
| L1 cache reference | 0.5 ns | ~1 ns | A few CPU cycles |
| Branch mispredict | 5 ns | ~3–5 ns | |
| L2 cache reference | 7 ns | ~3–5 ns | |
| Mutex lock/unlock (uncontended) | 25 ns | ~15–25 ns | Contended locks are far slower |
| Main memory reference | 100 ns | ~80–100 ns | Has barely improved in 15 years |
| Compress 1 KB (Snappy/LZ4 class) | 3 µs | ~1–2 µs | |
| Send 1 KB over the network | 10 µs at 1 Gbps | ~1 µs at 10 Gbps | Serialization time only, not latency |
| Random 4 KB read from SSD | 150 µs | ~20–100 µs (NVMe) | Cloud network block storage: ~0.5–1 ms |
| Read 1 MB sequentially from memory | 250 µs | tens of µs | ~10–30 GB/s per core |
| Round trip within a datacenter | 500 µs | ~100–500 µs | Same zone; cross-zone ~0.5–2 ms |
| Read 1 MB sequentially from SSD | 1 ms | ~0.2–0.3 ms (NVMe) | 3–7 GB/s |
| Disk (HDD) seek | 10 ms | ~4–10 ms | Unchanged: it is mechanical |
| Read 1 MB sequentially from HDD | 20 ms | ~5 ms | ~200 MB/s |
| Round trip US East ↔ US West | ~60–70 ms | ||
| Round trip California ↔ Netherlands | 150 ms | ~140–150 ms | Bounded by physics |
The speed of light sets the floor. Light in fibre travels at about m/s, two thirds of its vacuum speed: about 5 µs per km one way, or 1 ms of round trip per 100 km. New York to London is about 5,600 km in a straight line, so at least 56 ms round trip before any routing or processing; real paths are longer. No amount of engineering removes this, which is why global systems replicate data close to users instead of making users call home.
Worked example. A page needs 30 database queries. Inside one zone at ~0.5 ms per round trip, issuing them sequentially costs 15 ms of pure network waiting. If the database is in another region 70 ms away, the same code costs 2.1 seconds. Batch them into one query, or move compute next to the data.
Time Constants
| Duration | Seconds | Useful For |
|---|---|---|
| 1 minute | 60 | Request rates |
| 1 hour | 3,600 | Batch processing |
| 1 day | 86,400 | Daily aggregates |
| 1 month | 2,592,000 | Monthly storage |
| 1 year | 31,536,000 | Capacity planning |
TIP
The 86,400 trick: A day has 86,400 seconds. Round to for quick math. This means:
- 1 QPS = ~100K requests/day
- 1K QPS = ~100M requests/day
- 10K QPS = ~1B requests/day
Rounding 86,400 up to understates QPS by about 14%. That is fine, but say so, and remember traffic is not spread evenly over 24 hours anyway.
QPS (Queries Per Second) Calculations
From Daily Active Users
Example: Twitter-like service
Peak vs Average
Production systems must handle peak load, not just average. Common rule of thumb:
The multiplier depends on how concentrated usage is. A global consumer app spread across time zones may peak at 2x; a single-country app, a sports app on match day, or a ticketing site at on-sale time can see 10–100x. When you can, take the peak from real data: the busiest minute of the busiest day.
From Concurrent Users
Correction (2026-11-09)
This formula is Little's Law rearranged, and it only holds when "concurrent users" means requests in flight. Real users spend most of their time reading, not waiting on a request. With concurrent users who each issue a request, wait seconds for it and then think for seconds, the throughput is . For 100,000 users with s and s, that is about 3,300 QPS, not the 500,000 the unmodified formula gives.
TypeScript (estimation helper):
interface EstimationParams {
dau: number;
requestsPerUserPerDay: number;
peakMultiplier?: number;
}
function estimateQPS(params: EstimationParams): {
averageQPS: number;
peakQPS: number;
dailyRequests: number;
} {
const { dau, requestsPerUserPerDay, peakMultiplier = 3 } = params;
const dailyRequests = dau * requestsPerUserPerDay;
const averageQPS = dailyRequests / 86_400;
const peakQPS = averageQPS * peakMultiplier;
return {
averageQPS: Math.round(averageQPS),
peakQPS: Math.round(peakQPS),
dailyRequests,
};
}
// Example: Social media
const result = estimateQPS({
dau: 300_000_000,
requestsPerUserPerDay: 100,
peakMultiplier: 3,
});
// { averageQPS: 347222, peakQPS: 1041667, dailyRequests: 30000000000 }
Python:
from dataclasses import dataclass
@dataclass
class QPSEstimate:
average_qps: int
peak_qps: int
daily_requests: int
def estimate_qps(
dau: int,
requests_per_user_per_day: int,
peak_multiplier: float = 3.0
) -> QPSEstimate:
daily_requests = dau * requests_per_user_per_day
average_qps = daily_requests / 86_400
peak_qps = average_qps * peak_multiplier
return QPSEstimate(
average_qps=round(average_qps),
peak_qps=round(peak_qps),
daily_requests=daily_requests
)
# Example
result = estimate_qps(dau=300_000_000, requests_per_user_per_day=100)
print(f"Average: {result.average_qps:,} QPS")
print(f"Peak: {result.peak_qps:,} QPS")
# Average: 347,222 QPS
# Peak: 1,041,667 QPS
Storage Calculations
General Formula
Add a factor for indexes and overhead (often 1.3–2x) and for headroom (disks should not run above ~70–80% full; LSM compaction and rebalancing need free space).
Worked Example: URL Shortener
Assumptions:
- 100M new URLs per day
- Each URL record: 500 bytes (short URL + long URL + metadata)
- Retention: 5 years
- Replication factor: 3
Worked Example: Chat Application
| Parameter | Value |
|---|---|
| DAU | 50M |
| Messages per user per day | 40 |
| Average message size | 200 bytes |
| Media messages (10% with 200KB avg) | 200 KB |
| Retention | Forever |
Note how media is 99% of the bytes. Text could live in a database; media belongs in object storage behind a CDN, with only a pointer in the message row. Finding the term that dominates is the point of the estimate.
Bandwidth Calculations
Example: If your service handles 10K QPS and each response averages 10 KB:
Bandwidth Conversion
| Unit | Equivalent |
|---|---|
| 1 Byte | 8 bits |
| 1 MB/s | 8 Mbps |
| 1 Gbps | 125 MB/s |
| 10 Gbps | 1.25 GB/s |
Separate ingress (uploads, writes) from egress (downloads, reads). They have different costs (cloud egress is billed, ingress usually is not) and different bottlenecks. A daily volume converts to an average rate via .
Memory Estimation
Intuition. You rarely cache everything. Access is skewed, so a small hot fraction of the data serves most requests. Memory sizing is: how big is the working set, and what does each entry cost including overhead?
Per-entry overhead is real: a Redis key with a small string value costs on the order of 50–100 bytes beyond the raw bytes (dictionary entry, object headers, allocator rounding). For a million 100-byte values, budget roughly 150–200 MB, not 100 MB.
Worked example. Session store for 100M users, 20% active on a given day, 1 KB per session: GB, plus overhead and a replica, so about 50 GB: a small Redis cluster. Also check connections and per-connection buffers (a WebSocket gateway holding 15 million connections at ~20 KB each needs 300 GB of RAM just for connection state).
Five Complete Worked Estimates
Each estimate lists assumptions first, then derives QPS, storage, bandwidth, memory and machines. The assumptions are illustrative round numbers, not any company's real figures; state yours out loud in an interview.
1. Twitter-Like Social Feed
| Assumption | Value |
|---|---|
| DAU | 300M |
| New tweets per day | 500M |
| Timeline tweets viewed per user per day | 100 |
| Tweet size (text + metadata) | ~1 KB stored, 300 bytes of text |
| Tweets with an image | 10%, ~200 KB each |
| Average followers per tweet author | 200 |
| Images viewed per user per day | 20 |
QPS.
At a 3x peak: about 17,000 writes/s and 1 million timeline-tweet reads/s. The read:write ratio is about 60:1.
Storage. Tweets: GB/day, about 180 TB/year before replication. Images: TB/day, 3.65 PB/year. Media dominates again.
Fan-out-on-write. Writing each tweet ID into every follower's home timeline means timeline inserts per day, about 1.16 million per second on average. At 8 bytes per tweet ID that is 800 GB/day of timeline writes. Capping each cached timeline at 800 entries bounds the memory: TB of RAM, a few dozen large cache nodes. Accounts with millions of followers break fan-out-on-write (one tweet becomes millions of writes), which is why such designs merge celebrity tweets at read time instead.
Bandwidth. Timeline reads: MB/s, about 2.8 Gbps. Images: PB/day, about 111 Gbps on average, which is a CDN problem, not an application-server problem.
Servers. At ~10,000 timeline reads per second per server (served from cache), 1 million peak reads/s needs about 100 servers, plus 2x for redundancy and deploys.
2. URL Shortener, End to End
| Assumption | Value |
|---|---|
| New short URLs per day | 100M |
| Read:write ratio | 100:1 |
| Record size | 500 bytes |
| Retention | 5 years |
- Write QPS: /s; peak about 3,500/s.
- Read QPS: 100x, about 116,000/s; peak about 350,000/s.
- ID space: 5 years is URLs. Base-62 with 6 characters gives , not enough; 7 characters give , about 19 times the need. Random 7-character codes need collision checks (the birthday bound bites at about 2.2 million codes; see Probability for System Design).
- Storage: 273 TB with 3x replication (computed above).
- Cache: reads concentrate on recent links. Caching the hottest 20% of the last 30 days' links: GB, a handful of cache nodes, which should absorb well over 90% of reads.
- Bandwidth: MB/s egress, under 0.5 Gbps. Tiny.
The conclusion an interviewer wants: this is a read-heavy, cache-friendly, storage-moderate system. The database sees about 1,200 writes/s and only cache misses for reads; the hard parts are ID generation and redirect latency, not throughput.
3. Chat Application
Same assumptions as the storage example above (50M DAU, 40 messages/user/day, 200-byte text, 10% media at 200 KB), plus: 30% of DAU online at peak, average group size 5.
- Message QPS: /s; peak about 70,000/s.
- Deliveries: each message fans out to about 5 recipients in groups, so up to 350,000 deliveries/s at peak, each a push over a persistent connection.
- Connections: million concurrent WebSocket connections. At ~100,000 connections per gateway (a practical figure, limited by memory and TLS state), that is 150 gateways plus spare. At ~20 KB of state per connection, 300 GB of RAM across them.
- Storage: 400 GB/day text plus 40 TB/day media (computed above).
- Bandwidth: text MB/s, negligible; media ingress 40 TB/day MB/s Gbps average, with downloads multiplied by group size.
The bottleneck is connection count and fan-out, not bytes. See the chat schema design page for the data model.
4. Video Upload and Streaming Platform
| Assumption | Value |
|---|---|
| Video uploaded | 500 hours per minute (the figure YouTube has published since 2019) |
| Average video length | 10 minutes |
| Upload bitrate (1080p source) | ~5 Mbps |
| Encoding ladder | 1080p 5 + 720p 2.5 + 480p 1 + 360p 0.7 + 240p 0.4 = 9.6 Mbps total |
| Hours watched per day | 1 billion (YouTube's 2017 figure) |
| Average delivered bitrate | 2.5 Mbps |
- Uploads: 500 hours/min = 720,000 hours/day = 4.3 million videos/day = 50 uploads per second. Modest QPS, enormous bytes.
- Ingest: PB/day of source video.
- Stored renditions: PB/day, about 9.3 PB/day with 3x replication (or about 4.7 PB/day with erasure coding at 1.5x overhead).
- Transcoding: if one CPU core needs about 2 hours to transcode one hour of source into the full ladder (an assumption; real figures vary hugely with codec and preset), that is 1.44 million core-hours per day: 60,000 cores running continuously. Codec choice is a compute-versus-bandwidth trade: AV1 saves bytes and costs many times more CPU to encode.
- Egress: EB/day, about 104 Tbps on average. Only a CDN with servers inside ISPs can carry that; this is the entire business.
5. Metrics and Monitoring System
| Assumption | Value |
|---|---|
| Hosts | 10,000 |
| Active time series per host | 1,000 |
| Scrape interval | 15 s |
| Raw sample | 16 bytes (8-byte timestamp + 8-byte float) |
| Compressed sample | ~1.37 bytes (Gorilla encoding, Facebook's VLDB 2015 paper) |
| Raw retention | 15 days |
- Active series: .
- Ingest rate: samples/s.
- Storage: samples/day; raw 920 GB/day, compressed about 79 GB/day. Fifteen days is about 1.2 TB, before replication. Compression is a 12x lever.
- Memory: the in-memory head block and index cost a few KB per active series in Prometheus-style engines; at ~4 KB that is about 40 GB.
- Long-term: downsample to 5-minute rollups for a year: points; at a few bytes each, a few TB.
Cardinality is the killer. Add a user_id label with 100,000 values to a metric present on every host and the series count can grow by up to 100,000x; nothing in this estimate survives that. Per-user data belongs in logs or traces, not metric labels. See Time-Series Databases.
Little's Law and Utilization
Little's Law
Intuition. In any stable system, the average number of things inside equals the rate they arrive times how long each stays. A shop with 10 customers arriving per minute who each stay 3 minutes has 30 customers inside on average.
It holds for any arrival pattern and any service-time distribution, as long as the system is stable (arrivals do not outpace departures forever).
Worked examples.
- Concurrency: 2,000 requests/s at 150 ms average latency means requests in flight. With 50 worker threads per instance, you need at least 6 instances, before headroom.
- Connection pools: 5,000 queries/s at 4 ms each keeps connections busy on average. A pool of 20 is exactly saturated and will queue constantly; size for the peak and for latency spikes (if query latency rises to 40 ms during a slow period, the same load needs 200 connections). See Connection Pooling.
- Queues: a Kafka consumer group with 30,000 messages of lag draining at 1,000 messages/s has 30 seconds of delay.
Utilization vs Latency
Intuition. A server that is busy 90% of the time is idle only 10% of the time, so a new request is likely to find a queue. As utilization approaches 1, queues grow without bound. For the simplest model (M/M/1: random arrivals, random service times, one server), the average time in system is
where is the service time. With ms:
| Utilization | Average latency |
|---|---|
| 50% | 20 ms |
| 70% | 33 ms |
| 80% | 50 ms |
| 90% | 100 ms |
| 95% | 200 ms |
| 99% | 1,000 ms |
Going from 80% to 90% busy doubles latency; 90% to 99% multiplies it by ten. This curve is why services target 50–70% utilization at peak, why "the CPU is only at 85%" can coexist with terrible latency, and why a small traffic increase near saturation causes a large latency jump. Multi-server pools (M/M/c) behave better at the same utilization, which is one argument for fewer, larger pools over many small ones. The full treatment is on Queueing Theory.
def mm1_latency(service_ms: float, utilization: float) -> float:
"""Average time in an M/M/1 system."""
if utilization >= 1:
return float("inf")
return service_ms / (1 - utilization)
def instances_needed(peak_rps: float, service_ms: float, workers_per_instance: int,
target_util: float = 0.6) -> int:
"""Little's Law concurrency, then divide by capacity at the target utilization."""
in_flight = peak_rps * service_ms / 1000
return int(-(-in_flight // (workers_per_instance * target_util))) # ceiling
for rho in (0.5, 0.8, 0.9, 0.99):
print(f"utilization {rho:.0%}: {mm1_latency(10, rho):7.1f} ms")
print("instances for 6,000 rps at 150 ms, 50 workers each, 60% target:",
instances_needed(6_000, 150, 50))
# utilization 50%: 20.0 ms
# utilization 80%: 50.0 ms
# utilization 90%: 100.0 ms
# utilization 99%: 1000.0 ms
# instances for 6,000 rps at 150 ms, 50 workers each, 60% target: 30
Availability Math and Nines Budgets
Composing Availability
A request path that needs every component is a series system; availability multiplies. Redundant copies form a parallel system; unavailability multiplies.
Worked example. A service promising 99.99% calls 10 dependencies serially, each also 99.99%: . It cannot meet its own SLO, even with perfect code. Either the dependencies must be better than the service's own target, or calls must be made non-critical (cached fallbacks, timeouts with degraded responses, asynchronous work).
Nines as Budgets
| Availability | Downtime per year | Per 30-day month | Error budget at 1B requests/month |
|---|---|---|---|
| 99% | 3.65 days | 7.2 hours | 10M failed requests |
| 99.9% | 8.77 hours | 43.2 minutes | 1M |
| 99.95% | 4.38 hours | 21.6 minutes | 500K |
| 99.99% | 52.6 minutes | 4.3 minutes | 100K |
| 99.999% | 5.26 minutes | 26 seconds | 10K |
Budgets drive policy. At 99.95% you have 21.6 minutes a month. If a bad deploy typically costs 10 minutes before rollback, you can afford two bad deploys a month, which tells you how much to invest in canarying and automatic rollback. At 99.99% (4.3 minutes) no human-in-the-loop recovery fits; detection and failover must be automatic.
Two more rules of thumb. Availability from MTBF and MTTR is , so halving repair time is as good as doubling time between failures. And all the parallel math assumes independent failures; correlated failures (a shared zone, a shared config push) usually dominate real outages. Both are derived on Probability for System Design.
Replication Factor Calculations
Write Amplification
With a replication factor of :
In other words, in single-leader replication the leader handles every write, so adding followers scales reads but not writes; to scale writes you shard. Storage engines add their own amplification on top: an LSM tree may rewrite each byte 10–30 times through compaction, so 100 MB/s of logical writes at RF 3 can mean several GB/s of physical disk writes across the cluster.
Quorum Reads and Writes
For a system with replicas, a write quorum , and a read quorum :
Common configurations:
| Config | Trade-off | |||
|---|---|---|---|---|
| Strong consistency | 3 | 2 | 2 | Balanced |
| Write-optimized | 3 | 1 | 3 | Fast writes, slow reads |
| Read-optimized | 3 | 3 | 1 | Slow writes, fast reads |
| Eventual consistency | 3 | 1 | 1 | Fast but stale reads possible |
The overlap condition is pigeonhole arithmetic: if , any read set and any write set share at least one replica, so a read sees the latest acknowledged write. (Strictly, "strong consistency" also needs correct handling of concurrent writes, sloppy quorums and failed writes; Quorums and Leaderless Replication covers the edge cases.)
Quorum latency is an order statistic. A quorum operation finishes when the -th (or -th) fastest replica answers. With and , you wait for the second-fastest of three, which is faster than waiting for a single specific replica's tail, because one slow replica is simply ignored. With you wait for the slowest. That is why hurts tail latency and availability at once.
Quorum availability is a binomial tail. With 3 replicas each up 99.9% of the time, succeeds with probability ; only .
Durability: Why Repair Time Matters
Intuition. With 3 replicas, data is lost only if all three copies die before the system re-replicates. The danger window is the repair time .
For a disk annual failure rate (on the order of 1–2% for modern drives in large fleets, per Backblaze's published drive statistics) and hourly failure rate , a rough estimate per chunk per year is
Worked example. AFR 2%, hour: per chunk-year. With chunks, about expected loss events per year. With hours: per chunk, or 0.36 loss events per year, almost 600 times worse, because the risk scales with . Fast, parallel re-replication (many nodes each copying a little) matters more than a fourth replica. Correlated failures (a power event taking out a rack) make all of this worse; that is what rack- and zone-aware placement is for.
Availability Calculation
For independent replicas, each with availability :
| Single node availability | 2 replicas | 3 replicas | 5 replicas |
|---|---|---|---|
| 99% (3.65 days downtime) | 99.99% | 99.9999% | 99.99999999% |
| 99.9% (8.77 hrs downtime) | 99.9999% | ~100% | ~100% |
| 99.99% (52.6 min downtime) | ~100% | ~100% | ~100% |
WARNING
These calculations assume independent failures. In practice, correlated failures (data center outages, shared dependencies) make real availability lower. This is why multi-region deployment matters.
This table is for "any one replica can serve" (reads from any copy). For majority quorums, use the binomial tail above instead: 3 nodes at 99% give 99.97%, not 99.9999%.
Consistent Hashing Math
Consistent hashing maps both keys and servers onto a ring of size (typically or for MD5/SHA-1). Each key is assigned to the next server clockwise on the ring.
graph TD
subgraph "Hash Ring"
direction TB
S1["Server A<br/>(position 30)"]
S2["Server B<br/>(position 120)"]
S3["Server C<br/>(position 220)"]
K1["Key 1<br/>(hash: 50) → B"]
K2["Key 2<br/>(hash: 150) → C"]
K3["Key 3<br/>(hash: 250) → A"]
end
Virtual Nodes
With physical servers, each server gets virtual nodes on the ring. This improves load distribution.
With virtual nodes per server, the standard deviation of load drops to about 5-10% of the mean — acceptable for most systems.
Why . Each server's share of the ring is the sum of arc lengths. With one point per server, arcs are exponentially distributed and wildly uneven: the standard deviation of a server's share is about equal to its mean. Summing independent arcs shrinks the relative spread by , the same law as in sampling. A simulation with 10 servers (MD5 positions) confirms it:
| Virtual nodes | Std dev of load (fraction of mean) | Busiest server | Lightest server |
|---|---|---|---|
| 1 | 94% | 3.26x mean | 0.12x |
| 10 | 29% | 1.29x | 0.52x |
| 100 | 7.2% | 1.12x | 0.87x |
| 1,000 | 3.0% | 1.04x | 0.93x |
import hashlib
import numpy as np
def ring_load_shares(n_servers: int, vnodes: int) -> np.ndarray:
"""Fraction of the hash ring owned by each server (1.0 = fair share)."""
points = sorted(
(int.from_bytes(hashlib.md5(f"node{s}:{i}".encode()).digest()[:8], "big"), s)
for s in range(n_servers) for i in range(vnodes)
)
share = np.zeros(n_servers)
ring = 2**64
for idx, (pos, server) in enumerate(points):
prev = points[idx - 1][0] - (ring if idx == 0 else 0)
share[server] += (pos - prev) / ring # arc from the previous point is owned by this one
return share * n_servers
for v in (1, 10, 100, 1000):
s = ring_load_shares(10, v)
print(f"V={v:>4}: std={s.std():.3f} max={s.max():.2f} min={s.min():.2f}")
# V= 1: std=0.942 max=3.26 min=0.12
# V= 10: std=0.285 max=1.29 min=0.52
# V= 100: std=0.072 max=1.12 min=0.87
# V=1000: std=0.030 max=1.04 min=0.93
The busiest server, not the average, sets capacity: at you must provision every server for 1.29x the mean, wasting 29% of the fleet. Virtual nodes cost memory for the ring and make membership changes touch more ranges, so 100–256 per server is a common compromise.
Rebalancing on Server Addition/Removal
When a server is added, only keys need to be remapped (where is total keys, is total servers). Compare this to modular hashing where nearly all keys must move:
Correction (2026-11-09)
Strictly, adding a server to existing ones moves about keys (the new server's fair share), as the table below says; is the figure for removing one of servers. For large the two are nearly equal.
| Event | Consistent Hashing | Modular Hashing |
|---|---|---|
| Add 1 server ( to ) | keys move | keys move |
| Remove 1 server | keys move | keys move |
Worked example. 10 servers holding 1 billion keys. Adding an 11th: consistent hashing moves about 91 million keys; modular hashing (hash % N) moves about 909 million, and every one of those is a cache miss or a data copy at the same moment.
TypeScript:
import { createHash } from "crypto";
class ConsistentHash {
private ring: Map<number, string> = new Map();
private sortedKeys: number[] = [];
private virtualNodes: number;
constructor(virtualNodes = 150) {
this.virtualNodes = virtualNodes;
}
private hash(key: string): number {
const h = createHash("md5").update(key).digest();
return h.readUInt32BE(0);
}
addServer(server: string): void {
for (let i = 0; i < this.virtualNodes; i++) {
const virtualKey = this.hash(`${server}:${i}`);
this.ring.set(virtualKey, server);
this.sortedKeys.push(virtualKey);
}
this.sortedKeys.sort((a, b) => a - b);
}
removeServer(server: string): void {
for (let i = 0; i < this.virtualNodes; i++) {
const virtualKey = this.hash(`${server}:${i}`);
this.ring.delete(virtualKey);
}
this.sortedKeys = this.sortedKeys.filter((k) => this.ring.has(k));
}
getServer(key: string): string | undefined {
if (this.sortedKeys.length === 0) return undefined;
const h = this.hash(key);
// Binary search for next server clockwise
let lo = 0, hi = this.sortedKeys.length;
while (lo < hi) {
const mid = (lo + hi) >> 1;
if (this.sortedKeys[mid] < h) lo = mid + 1;
else hi = mid;
}
const idx = lo % this.sortedKeys.length;
return this.ring.get(this.sortedKeys[idx]);
}
}
Python:
import hashlib
from bisect import bisect_right
class ConsistentHash:
def __init__(self, virtual_nodes: int = 150):
self.virtual_nodes = virtual_nodes
self.ring: dict[int, str] = {}
self.sorted_keys: list[int] = []
def _hash(self, key: str) -> int:
h = hashlib.md5(key.encode()).digest()
return int.from_bytes(h[:4], "big")
def add_server(self, server: str) -> None:
for i in range(self.virtual_nodes):
virtual_key = self._hash(f"{server}:{i}")
self.ring[virtual_key] = server
self.sorted_keys = sorted(self.ring.keys())
def remove_server(self, server: str) -> None:
for i in range(self.virtual_nodes):
virtual_key = self._hash(f"{server}:{i}")
self.ring.pop(virtual_key, None)
self.sorted_keys = sorted(self.ring.keys())
def get_server(self, key: str) -> str | None:
if not self.sorted_keys:
return None
h = self._hash(key)
idx = bisect_right(self.sorted_keys, h)
if idx == len(self.sorted_keys):
idx = 0
return self.ring[self.sorted_keys[idx]]
# Check the rebalancing claim: adding an 11th server should move about 1/11 of keys
ch = ConsistentHash()
for s in range(10):
ch.add_server(f"server-{s}")
keys = [f"user:{i}" for i in range(100_000)]
before = {k: ch.get_server(k) for k in keys}
ch.add_server("server-10")
moved = sum(before[k] != ch.get_server(k) for k in keys)
print(f"moved {moved / len(keys):.1%} of keys (ideal 1/11 = {1 / 11:.1%})")
# moved 9.4% of keys (ideal 1/11 = 9.1%)
Sharding Math: Hot Partitions
Intuition
Hashing spreads keys evenly. It does nothing for traffic when a few keys are far more popular than the rest. Real popularity is often Zipf-like: the -th most popular key gets traffic proportional to with near 1.
With keys and , the most popular key's share of all traffic is
Worked Example
One million keys, Zipf with : , so the top key gets 6.9% of all traffic, the top 10 keys 20%, and the top 1,000 keys 52%. Spread across 100 shards (average 1% each), the shard holding the top key carries about 7.9% of traffic: 7.8 times the average (simulated with random key placement). Adding shards does not help: the top key's 6.9% is on one shard no matter how many there are.
import numpy as np
rng = np.random.default_rng(0)
n_keys, n_shards = 1_000_000, 100
weights = 1 / np.arange(1, n_keys + 1) # Zipf, s = 1
weights /= weights.sum()
shard_of_key = rng.integers(0, n_shards, n_keys) # hash placement
load = np.bincount(shard_of_key, weights=weights, minlength=n_shards)
print(f"top key share: {weights[0]:.1%}")
print(f"hottest shard: {load.max() / load.mean():.1f}x average, coolest: {load.min() / load.mean():.2f}x")
# top key share: 6.9%
# hottest shard: 7.8x average, coolest: 0.50x
Remedies. Cache the hot keys in front of the shards (the cache absorbs exactly the skew that hurts); replicate hot keys across several shards for reads; split hot keys for writes by appending a suffix (counter:42#0 … counter:42#9) and summing on read; and detect hot keys from live traffic sampling rather than guessing. DynamoDB's adaptive capacity and "split for heat" exist because of this arithmetic. See Sharding.
Shard count rule of thumb. Size so that the hottest shard at peak stays under its limit, not the average one, and keep enough shards that one shard's failure or rebalancing touches a small fraction of data. Many systems pre-split into many more logical shards (hundreds or thousands) than physical nodes so that rebalancing moves whole logical shards.
Cache Hit Ratio and Backend Load
Intuition
The backend only sees misses. So the number that matters is not the hit ratio but the miss ratio, and small changes in hit ratio near 100% are huge changes in miss ratio.
The 90% → 99% Insight
| Hit ratio | Backend load (of total) | Average latency (1 ms hit, 50 ms miss) |
|---|---|---|
| 90% | 10% | 5.9 ms |
| 95% | 5% | 3.5 ms |
| 99% | 1% | 1.5 ms |
| 99.9% | 0.1% | 1.05 ms |
Going from 90% to 99% sounds like a 10% improvement. It is a 10x reduction in database load and a 4x reduction in average latency. Conversely, a cache that drops from 99% to 90% (a bad deploy that changes key formats, an eviction storm, a cold restart) multiplies backend load by 10 instantly. If the database was sized for the 99% world, it falls over.
Worked example. 100,000 reads/s with a 99% hit ratio: the database serves 1,000 reads/s, comfortable for one primary and a replica. Restart the cache fleet and the hit ratio briefly goes to 0%: the database receives 100,000 reads/s, 100 times its normal load. This is why cache restarts are rolled, why caches are warmed before taking traffic, and why request coalescing exists. See Cache Sizing Math and Thundering Herd.
def backend_qps(total_qps: float, hit_ratio: float) -> float:
return total_qps * (1 - hit_ratio)
for h in (0.90, 0.99, 0.999):
print(f"hit {h:.1%}: backend sees {backend_qps(100_000, h):>8,.0f} QPS")
print(f"cache cold (0%): backend sees {backend_qps(100_000, 0):,.0f} QPS")
# hit 90.0%: backend sees 10,000 QPS
# hit 99.0%: backend sees 1,000 QPS
# hit 99.9%: backend sees 100 QPS
# cache cold (0%): backend sees 100,000 QPS
Probabilistic Data-Structure Sizing
When exact answers would need memory proportional to the data, a small, tunable error buys enormous savings.
Bloom Filter
Intuition. A bit array plus hash functions answers "definitely not present" or "probably present". It never gives false negatives; the false-positive rate is set by bits per element.
Worked example. A crawler tracking 1 billion seen URLs at a 1% false-positive rate needs bits per URL: about 1.2 GB with . Storing the URLs themselves (about 70 bytes average) would take roughly 70 GB. Each tenfold reduction in costs another 4.8 bits per element. A false positive here means skipping a URL that was never crawled, which is acceptable; for "has this password been breached", a false positive needs a confirming lookup. Derivation and variants: Bloom Filters.
HyperLogLog
Intuition. To count distinct items, hash each one and watch for unusually long runs of leading zeros: seeing a hash starting with 20 zeros suggests you have seen around distinct items. Averaging this over independent buckets (registers) tames the noise.
Worked example. Redis's HyperLogLog uses registers of 6 bits: 12 KB per counter, with standard error , whether it counts a thousand or a billion items. Counting daily unique visitors for 10 million pages exactly would need a set per page; with HLL it is at most 120 GB (less with Redis's sparse encoding for small counts), and HLLs merge, so weekly uniques are the union of seven daily sketches without rescanning.
Count-Min Sketch
For "how often has this item appeared" (heavy hitters, hot-key detection), a count-min sketch with width and depth overestimates a count by at most with probability . With and : , , about 13,600 counters, or 54 KB at 4 bytes each, regardless of how many distinct items flow through.
| Structure | Answers | Memory example | Error |
|---|---|---|---|
| Bloom filter | Is in the set? | 1.2 GB for 1B items | 1% false positives, no false negatives |
| HyperLogLog | How many distinct? | 12 KB per counter | ~0.81% standard error |
| Count-min sketch | How often did appear? | 54 KB | Overestimates by at most 0.1% of the total |
import math
def bloom_size(n: int, p: float) -> tuple[float, int]:
bits = -n * math.log(p) / math.log(2) ** 2
return bits / 8 / 1e9, round(bits / n * math.log(2))
gb, k = bloom_size(1_000_000_000, 0.01)
print(f"Bloom filter, 1B items at 1%: {gb:.2f} GB, k={k}")
print(f"HyperLogLog, 16384 registers: {16384 * 6 // 8:,} bytes, error {1.04 / math.sqrt(16384):.2%}")
w, d = math.ceil(math.e / 0.001), math.ceil(math.log(1 / 0.01))
print(f"Count-min sketch eps=0.001 delta=0.01: {w} x {d} = {w * d * 4 / 1024:.0f} KB")
# Bloom filter, 1B items at 1%: 1.20 GB, k=7
# HyperLogLog, 16384 registers: 12,288 bytes, error 0.81%
# Count-min sketch eps=0.001 delta=0.01: 2719 x 5 = 53 KB
Tail-Latency Amplification
Intuition. A request that waits on many backends is as slow as the slowest. If each backend is slow with probability (say for "slower than its p99"), a request touching of them in parallel is slow with probability
Worked example. Fan-out of 100: of requests hit at least one backend's p99. The backend's p99 has become the user's median. To keep the user-facing p99 at the backend's p99 with fan-out 100, each backend needs that latency at its p99.99. The effect also applies to sequential calls: 5 sequential services each with a 1% slow tail give slow requests.
Design consequences. Budget latency per hop (if the user SLO is 200 ms at p99 and the request makes 4 sequential calls, each gets roughly a 50 ms p99.9 target, not 50 ms p99); use hedged requests and partial results for wide fan-out; keep fan-out small on latency-critical paths. The derivation and a simulation are on Probability for System Design.
Cost Estimation
Intuition
Cost is the estimate that reaches finance, and it is usually dominated by one term. Multiply each resource by its unit price, find the biggest line, and design around it.
Approximate unit prices (AWS us-east-1 list prices as of 2025; they change, vary by region and commitment, and every provider differs, so check current pricing before relying on them):
| Resource | Approximate list price |
|---|---|
| Object storage (S3 Standard, first 50 TB) | USD 0.023 per GB-month |
| Block storage (EBS gp3) | USD 0.08 per GB-month |
| Internet egress (first 10 TB/month, falling with volume) | USD 0.09 per GB, about USD 0.05 per GB above 150 TB |
| General-purpose VM, 2 vCPU / 8 GB (m7i.large, on-demand) | about USD 0.10 per hour, USD 74 per month |
Worked Examples
- URL shortener storage: 273 TB on block storage at USD 0.08/GB-month is about USD 22,000 a month; as object storage it would be USD 6,300, but object storage is too slow for redirect lookups. The real choice is a database on SSD, so use the block-storage figure, and notice it is larger than the compute bill.
- Chat media: 40 TB/day is 14.6 PB/year. At USD 0.023/GB-month, a year of media costs about USD 336,000 per month to store by the end of the year, and the bill grows every month because retention is "forever". Infrequent-access and archive tiers (several times cheaper) for old media are the obvious lever.
- Egress: serving 1 PB a month from a cloud region at about USD 0.05/GB is about USD 50,000 a month, before CDN discounts. Egress is frequently the largest line for media products, and why CDNs and peering agreements matter.
- Compute: 100 servers of the m7i.large class is about USD 7,400 a month on demand; 1- to 3-year commitments typically cut that by roughly 30–60%.
The pattern: storage grows forever, egress scales with popularity, compute scales with peak. A cost estimate that ignores growth (storage at month 1 rather than month 36) is off by an order of magnitude. See FinOps cost optimization.
CAP Theorem: Formal Intuition
The CAP theorem states that a distributed system can provide at most two of three guarantees simultaneously:
- Consistency (C): Every read receives the most recent write
- Availability (A): Every request receives a response (not an error)
- Partition tolerance (P): The system continues to operate despite network partitions
Why You Must Choose
During a network partition, a message from node A cannot reach node B. You have two options:
- Choose Consistency (CP): Refuse to respond until the partition heals. You lose availability.
- Choose Availability (AP): Respond with potentially stale data. You lose consistency.
Since network partitions are inevitable in distributed systems, the real choice is between CP and AP during partition events.
graph TD
P["Network Partition<br/>Occurs"] --> Q{"Respond to<br/>client?"}
Q -->|"Yes, even with<br/>stale data"| AP["AP System<br/>(Cassandra, DynamoDB)"]
Q -->|"No, wait for<br/>consensus"| CP["CP System<br/>(HBase, MongoDB)"]
style AP fill:#ff9800,color:#fff
style CP fill:#2196f3,color:#fff
PACELC Extension
PACELC extends CAP: if there is a Partition, choose A or C; Else (normal operation), choose Latency or Consistency.
| System | During Partition | Normal Operation |
|---|---|---|
| DynamoDB | AP | EL (low latency) |
| MongoDB | CP | EC (consistent) |
| Cassandra | AP | EL (tunable) |
| PostgreSQL | CP | EC (ACID) |
Note (2026-11-09)
These labels depend on configuration. Abadi's original PACELC paper (IEEE Computer, 2012) classified MongoDB as PA/EC; with majority write concern and primary reads it behaves as PC/EC, as the table shows. DynamoDB with strongly consistent reads and Cassandra at QUORUM move toward C. Classify the configuration, not the product. See CAP Theorem.
Estimation Cheat Sheet
Common Service Numbers
Note (2026-11-09)
These rows are illustrative orders of magnitude under assumptions like those in the worked estimates above, not published figures. By this page's own Twitter-like estimate, ~300K is the average read QPS; the peak would be closer to 1M.
| Service | DAU | Peak QPS | Storage/day |
|---|---|---|---|
| Twitter-scale | 300M | ~300K | ~15 TB |
| Instagram-scale | 500M | ~500K | ~100 TB (images) |
| WhatsApp-scale | 2B | ~1M | ~50 TB |
| YouTube-scale | 2B | ~200K | ~500 TB (video) |
| URL shortener | 100M | ~10K | ~50 GB |
Server Capacity Rules of Thumb
| Resource | Single Server | Notes |
|---|---|---|
| QPS (web server) | 1,000-10,000 | Depends on response time |
| QPS (database) | 1,000-5,000 | With indexing |
| QPS (cache - Redis) | 100,000+ | In-memory |
| RAM | 64-256 GB | Commodity |
| Disk | 1-10 TB SSD | Fast I/O |
| Network | 1-10 Gbps | Standard NIC |
The database row is deliberately conservative: a well-indexed PostgreSQL or MySQL server on modern NVMe can serve tens of thousands of simple point reads per second, but complex queries, writes with many indexes, and lock contention pull real numbers far lower. Measure your own workload. Cloud instances commonly offer 10–25 Gbps or more today.
Quick Estimation Formulas
The Estimation Cheat Table
| Quantity | Formula | Handy number |
|---|---|---|
| Average QPS | daily requests / 86,400 | 1M/day ≈ 12/s; 1B/day ≈ 11,600/s |
| Peak QPS | 2–5 x average (more for spiky products) | |
| Bandwidth | QPS x payload | 1 TB/day ≈ 93 Mbps average |
| Storage | writes/day x size x days x replicas x overhead | 1 KB x 1B/day ≈ 1 TB/day |
| Concurrency | Little: | 1,000/s x 200 ms = 200 in flight |
| Queue latency | for M/M/1 | 90% busy = 10x service time |
| Series availability | 10 deps at 99.99% = 99.9% | |
| Parallel availability | 2 x 99.9% = 99.9999% if independent | |
| Backend load | x QPS | 90% → 99% hits = 10x less load |
| Fan-out tail | n = 100, p = 1% gives 63% | |
| Bloom filter | 1.44 log2(1/p) bits per item | 1% ≈ 9.6 bits |
| HyperLogLog | error ≈ 1.04/√m | 12 KB ≈ 0.81% |
| Speed of light in fibre | ~200 km per ms | 1 ms RTT per 100 km |
Common Estimation Mistakes
- Using average instead of peak. Capacity is set by the busiest minute. Average QPS times a stated peak factor, or the measured peak.
- Forgetting replication, indexes and headroom. Logical bytes x 3 replicas x 1.5 overhead / 0.7 fill factor is about 6.4 times the naive number.
- Mixing bits and bytes. Network is quoted in bits per second, storage in bytes. 1 Gbps is 125 MB/s, not 1 GB/s.
- Confusing GB and GiB at scale. 10% at the TB level, 13% at the PB level.
- Assuming uniform load. Hot keys, celebrity accounts and diurnal peaks break per-shard averages. Size the hottest shard.
- Assuming independent failures. Three replicas in one zone are one replica with extra steps.
- Running servers near 100% utilization. Latency is ; plan for 50–70% at peak.
- Reading "QPS = users / latency" as user count. It is in-flight requests; real users mostly think.
- Ignoring growth. Storage at month 36, not month 1. Retention "forever" means cost grows forever.
- Precision theatre. "347,222 QPS" signals false accuracy; "about 350K, peak about 1M" is the right register.
Practice Estimation Problems
Correction (2026-11-09)
The YouTube row's variables did not produce its answer: 500 hours per minute is 720,000 hours (2.6 billion seconds) per day, and at 5 MB/s that is about 13 PB/day, not 2.5 PB. About 2.5 PB/day corresponds to roughly 1 MB/s (8 Mbps) per stored hour across all renditions; the row now shows both. The fan-out row now states its assumptions (200M tweets/day, ~30 bytes per timeline entry) so that "~3 TB" can be reproduced: .
| Problem | Key Variables | Expected Answer |
|---|---|---|
| "Estimate YouTube storage per day" | 500 hrs video/min, 5 MB/sec, resolutions | ~13 PB/day at 5 MB/s (~2.5 PB/day at ~1 MB/s) |
| "How many servers for 1M QPS?" | 5K QPS/server | ~200 servers |
| "Twitter fan-out storage" | 300M users, 500 avg followers, 200M tweets/day, ~30 B per timeline entry | ~3 TB write amplification/day |
| "Cache size for 100M users" | 20% active, 1 KB per session | ~20 GB |
TIP
When in doubt during an interview, round aggressively and state your assumptions clearly. " seconds in a day" is close enough and much easier to multiply than 86,400. Interviewers value clarity of reasoning over arithmetic precision.
Interview Questions
1. Estimate the QPS, storage and cache size for a URL shortener with 100M new links a day.
Answer. Writes /s (peak about 3,500); reads at 100:1, about 116,000/s (peak about 350,000). Storage at 500 bytes for 5 years: 91 TB, 273 TB with RF 3. 7-character base-62 codes give IDs, enough for the needed; random codes need collision checks. Cache the hot fraction of recent links, about 300 GB. Bandwidth under 0.5 Gbps. The system is read-heavy and cache-friendly; the database mostly sees writes and misses.
2. Your cache hit ratio improves from 90% to 99%. How much does database load change?
Answer. The database sees misses: 10% of traffic becomes 1%, a 10x reduction. Average latency with 1 ms hits and 50 ms misses goes from 5.9 ms to 1.5 ms. The flip side: a cold cache or key-format change that drops the hit ratio to 90% multiplies DB load by 10, so size and protect the DB (rolling restarts, warming, request coalescing) for that case.
3. A service handles 2,000 RPS at 150 ms average latency. How many concurrent requests, and what happens if latency doubles?
Answer. Little's Law: in flight. If a dependency slows and latency doubles to 300 ms at the same arrival rate, concurrency doubles to 600. If the thread or connection pool was sized at 400, requests now queue, latency grows further, and the system can spiral. Size pools for peak latency, and use timeouts and load shedding to cap concurrency.
4. Why do services target 60% utilization rather than 90%?
Answer. Queueing delay grows like . For M/M/1 with 10 ms service time, latency is 25 ms at 60%, 100 ms at 90% and 1 s at 99%. At 90% a small traffic spike or a slow host pushes the system into the steep part of the curve. Headroom also covers losing a zone (with 3 zones, surviving one loss needs each to run below about 67%).
5. A product promises 99.99% availability and calls 10 services in series, each at 99.99%. Is that possible?
Answer. No: . Either the dependencies must be much better (around 99.999% each), or the calls must be made non-critical: cache fallbacks, degraded responses, asynchronous processing, or parallel redundant paths. At 99.99% the monthly budget is 4.3 minutes, so recovery must also be automatic.
6. With 1 million keys following a Zipf distribution, will adding shards fix a hot shard?
Answer. No. The top key alone has about of traffic, all on one shard; with 100 shards that shard runs at about 7.8x average no matter how many shards exist. Fixes target the key: a cache tier in front, read replicas of hot keys, splitting hot write keys into sub-keys and summing on read, and hot-key detection from sampled traffic.
7. How much memory does a Bloom filter need for 1 billion URLs at a 1% false-positive rate? And to count unique visitors?
Answer. bits per item: about 1.2 GB with 7 hash functions, versus about 70 GB to store the URLs. For unique visitors, a HyperLogLog: 12 KB per counter at about 0.81% standard error, mergeable across days.
8. Your search aggregator fans out to 100 shards. Each shard's p99 is 40 ms. What is the user's p99?
Answer. of requests wait on at least one shard's p99, so 40 ms is roughly the user's median; the user p99 is set by the shards' p99.99. Mitigations: hedged requests after the shard p95, returning partial results, fewer and larger shards on the hot path, and fixing the shard's tail causes.
9. Data is stored with 3 replicas. Is a fourth replica or faster re-replication the better durability investment?
Answer. Loss requires all replicas to fail within the repair window , and the probability scales with for 3 replicas. Cutting repair time from 24 hours to 1 hour reduces loss probability about 576-fold (in the worked example, from 0.36 to 0.0006 expected loss events per year across a billion chunks). Parallel re-replication is usually cheaper than 33% more storage, though a fourth replica in another zone also guards against correlated failures, which this model ignores.
10. Estimate the monthly egress bill for a video app serving 1 PB a month, and what you would do about it.
Answer. At roughly USD 0.05 per GB at high-volume tiers, 1 PB ( GB) is about USD 50,000 a month (more at lower tiers). Levers: a CDN with negotiated rates or ISP-embedded caches, better codecs (fewer bytes per hour of video, traded against encoding CPU), adaptive bitrate so mobile users do not fetch 1080p, and caching popular content at the edge.
Further Reading
- Probability for System Design — collisions, reliability, tail latency, Monte Carlo
- Statistics for A/B Testing — measuring whether a change helped
- Queueing Theory — Little's Law, utilization and latency in depth
- Quorums and Leaderless Replication — N, W, R and their edge cases
- Consistent Hashing — full deep-dive on ring-based hashing
- Estimation Cheat Sheet — interview-focused numbers
- Estimation Practice — more worked problems
- Cache Sizing Math — working sets and hit ratios
- Databases — storage engine internals
- Greedy Algorithms — optimization under constraints
- Bit Manipulation — bloom filter math
- Advanced Data Structures — data structure trade-offs