On this page

The Network Layer: Routing, ICMP, DHCP & IPv6

The data link layer delivers frames between neighbours on the same link. The network layer (layer 3) delivers packets between any two hosts on the planet, across hundreds of links run by thousands of organisations that have never coordinated with each other. It does this with three ideas:

  1. A global address space. IP addresses are hierarchical, so a router can make a decision about four billion destinations from a table of a million prefixes.
  2. Hop-by-hop forwarding. Each router looks at the destination address, picks an outgoing interface, and forgets the packet. No router knows the whole path.
  3. Routing protocols. Distributed algorithms that fill those forwarding tables and repair them when links fail.

IP itself is best-effort. Packets can be lost, duplicated, reordered or corrupted, and nobody tells you except by ICMP, sometimes. Reliability is the transport layer's job (TCP/IP Deep Dive). This page covers what IP does guarantee and how it finds its way.

It assumes you're comfortable with CIDR and subnet masks (IP Addressing & Subnetting). For NAT, see NAT. For inter-domain routing in depth, see BGP.


The IPv4 Header, Field by Field

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Version|  IHL  |    DSCP   |ECN|          Total Length         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|         Identification        |Flags|     Fragment Offset     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|  Time to Live |    Protocol   |        Header Checksum        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                       Source Address                          |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                    Destination Address                        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                    Options (if IHL > 5)                       |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
FieldBitsPurpose and gotchas
Version44 for IPv4
IHL4Header length in 32-bit words. Minimum 5 (20 bytes), maximum 15 (60 bytes)
DSCP6Differentiated Services class (e.g. EF = 46 for voice). Often rewritten or ignored at network boundaries
ECN2Explicit Congestion Notification. Routers mark instead of dropping
Total Length16Header + data in bytes. Maximum 65,535
Identification16Same value on every fragment of one datagram, so the receiver can group them
Flags3Bit 0 reserved, DF (Don't Fragment), MF (More Fragments)
Fragment Offset13Where this fragment's data starts, in 8-byte units
TTL8Decremented by every router. At 0 the packet is dropped and ICMP Time Exceeded is sent back. Prevents infinite loops, and is how traceroute works
Protocol8What's inside: 1 ICMP, 6 TCP, 17 UDP, 47 GRE, 50 ESP, 89 OSPF
Header Checksum16One's complement sum over the header only. Recomputed at every hop because TTL changes
Source / Destination32 + 32The addresses. Unchanged end to end, unless NAT rewrites them
Options0–320Record route, timestamp, source routing. Rarely used, often filtered

Worked Example: Decoding a Header

Raw bytes from a capture:

45 00 00 73 00 00 40 00 40 11 b8 61 c0 a8 00 01 c0 a8 00 c7
BytesFieldDecoded
45Version / IHLVersion 4, IHL 5 → 20-byte header, no options
00DSCP / ECNBest effort, no ECN
00 73Total length0x73 = 115 bytes → 95 bytes of payload
00 00Identification0
40 00Flags / offset0100 0000 0000 0000 → DF = 1, MF = 0, offset 0. Not a fragment
40TTL64 (a Linux or macOS sender, likely zero or few hops away)
11Protocol17 → UDP
b8 61Checksum0xB861. The data-link page computes this exact value step by step
c0 a8 00 01Source192.168.0.1
c0 a8 00 c7Destination192.168.0.199

The checksum arithmetic is worked through in The Data Link Layer, in the checksum section.


Fragmentation

Every link has a Maximum Transmission Unit: 1500 bytes for Ethernet, 9001 inside an AWS VPC, 1492 for PPPoE, 1280 as the IPv6 minimum. When an IPv4 router must forward a datagram larger than the next link's MTU and DF is not set, it fragments:

  • Each fragment gets a copy of the IP header with the same Identification.
  • The data is split into chunks that are multiples of 8 bytes, except the last.
  • Fragment Offset = position of the chunk in the original data ÷ 8.
  • MF = 1 on every fragment except the last.
  • Only the destination reassembles. Routers along the way never do.

Worked Example: 4000-Byte Datagram over a 1500-Byte MTU

The original datagram is 4000 bytes total: a 20-byte header and 3980 bytes of data, ID = 777.

Max data per fragment = 1500 − 20 = 1480 (1480 / 8 = 185, a whole number, so no rounding is needed).

FragmentData bytesData rangeOffset field (÷8)MFTotal Length
114800–1479011500
214801480–295918511500
310202960–397937001040

Check: 1480 + 1480 + 1020 = 3980. ✔ Offset 370 × 8 = 2960. ✔

Worked Example: Refragmenting at a 576-Byte MTU

Fragment 1 above (1480 data bytes, offset 0, MF = 1) reaches a link with MTU 576.

Max data per piece = ⌊(576 − 20) / 8⌋ × 8 = ⌊69.5⌋ × 8 = 552 bytes.

PieceData bytesOffset fieldMFWhy MF
1a55201More pieces follow
1b552691More pieces follow
1c3761381The original fragment had MF = 1, so more data follows elsewhere

Fragment 3 (1020 bytes at offset 370, MF = 0) splits into 552 bytes at offset 370 (MF = 1) and 468 bytes at offset 439 (MF = 0, since it's the true end of the datagram).

Offsets are always relative to the original datagram, which is why the receiver can reassemble fragments of fragments in any order.

Why Fragmentation Is Avoided

  • Loss amplification. Lose any one fragment and the whole datagram is lost. With 3 fragments and 1% loss per packet, about 3% of datagrams are lost.
  • Reassembly cost and attacks. The receiver must buffer partial datagrams, with a timeout of 30 s on Linux. Teardrop and overlapping-fragment attacks have targeted that buffering, and fragment floods exhaust it.
  • Middleboxes. Only the first fragment carries the TCP/UDP ports. Firewalls, NAT and load balancers that hash on ports struggle with the rest, and many simply drop non-first fragments.
  • ID wraparound. At high rates the 16-bit ID repeats within the reassembly timeout, so pieces of different datagrams can be glued together.

Path MTU Discovery (PMTUD)

Instead of fragmenting, modern TCP stacks set DF = 1 on every packet and discover the smallest MTU on the path:

sequenceDiagram
    participant H as Host
    participant R1 as Router MTU 1500
    participant R2 as Router next link MTU 1400
    participant D as Destination
    H->>R1: 1500-byte packet, DF set
    R1->>R2: forwarded
    Note over R2: too big for the 1400 link, DF set, so drop
    R2->>H: ICMP type 3 code 4, fragmentation needed, next-hop MTU 1400
    Note over H: cache path MTU 1400 for D, lower TCP segment size
    H->>R1: 1400-byte packet, DF set
    R1->>R2: forwarded
    R2->>D: delivered

The PMTUD black hole. If a firewall drops all ICMP, the "fragmentation needed" message never arrives. Small packets (the TCP handshake, short requests) get through, and large ones vanish. The symptom is that SSH connects but hangs on ls of a large directory, or that HTTPS hangs after the TLS handshake. Fixes:

  • Allow ICMP type 3 code 4 (and ICMPv6 type 2, "Packet Too Big") through firewalls. Always.
  • TCP MSS clamping on the router at the tunnel or PPPoE edge: rewrite the MSS in SYN packets to fit (iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu).
  • Packetization-layer PMTUD (RFC 4821/8899), which probes with increasing sizes and needs no ICMP. On Linux, enable it with net.ipv4.tcp_mtu_probing=1.
# Test the path MTU by hand: 1472 bytes of ICMP data + 8 ICMP + 20 IP = 1500
ping -M do -s 1472 example.com     # "message too long" or silence means the path MTU is below 1500
ping -M do -s 1372 example.com     # binary search downward

tracepath example.com              # reports "pmtu 1400" at the hop where it shrinks
ip route get 93.184.216.34         # shows a cached "mtu 1400" after PMTUD has run

IPv6 never fragments in transit

IPv6 routers don't fragment. Too-big packets are dropped and ICMPv6 Packet Too Big is returned. Only the source can fragment, using an extension header. Blocking ICMPv6 breaks IPv6 far more thoroughly than blocking ICMP breaks IPv4.


Forwarding vs Routing

These two words get used interchangeably, but they are different jobs on different timescales.

Forwarding (data plane)Routing (control plane)
QuestionWhich interface does this packet go out?What should the forwarding table contain?
TimescaleNanoseconds per packetSeconds to minutes per topology change
WhereLine card ASIC with TCAM, or the kernel fast pathRouter CPU running OSPF, BGP, and so on
Data structureFIB (Forwarding Information Base)RIB (Routing Information Base): every candidate route from every protocol
AnalogyReading the road sign at a junctionDrawing the road map
flowchart LR
    subgraph CP["Control plane (router CPU)"]
        OSPF["OSPF"] --> RIB["RIB: all candidate routes"]
        BGP["BGP"] --> RIB
        STATIC["Static and connected routes"] --> RIB
        RIB -->|"best route per prefix (admin distance, metric)"| FIBW["install"]
    end
    subgraph DP["Data plane (ASIC or kernel)"]
        FIB["FIB: prefix to next hop and interface"]
        IN["Packet in"] --> LPM["Longest-prefix match"] --> OUT["Rewrite L2, decrement TTL, packet out"]
        FIB -.-> LPM
    end
    FIBW --> FIB

When several protocols offer a route to the same prefix, the router prefers the one with the lowest administrative distance. Cisco defaults: connected 0, static 1, eBGP 20, OSPF 110, IS-IS 115, RIP 120, iBGP 200. On Linux, the metric plays a similar role between routes in the same table.

Longest-Prefix Match

A forwarding table holds prefixes, not addresses, and several prefixes can contain the same destination. The rule is that the most specific (longest) matching prefix wins.

Worked example. A router's table:

PrefixNext hop
0.0.0.0/0ISP (203.0.113.1)
10.0.0.0/8R1
10.1.0.0/16R2
10.1.2.0/24R3
10.1.2.128/25R4
DestinationMatching prefixesLongestNext hop
10.1.2.130/0, /8, /16, /24, /25/25R4
10.1.2.50/0, /8, /16, /24 (50 < 128, so not the /25)/24R3
10.1.3.7/0, /8, /16/16R2
10.9.9.9/0, /8/8R1
8.8.8.8/0/0ISP

In binary, for 10.1.2.130 against 10.1.2.128/25, only the first 25 bits are compared:

10.1.2.130   = 00001010.00000001.00000010.1 0000010
10.1.2.128/25= 00001010.00000001.00000010.1 ───────   ← 25 bits match ✔
10.1.2.50    = 00001010.00000001.00000010.0 0110010   ← bit 25 differs ✘

Real routers do this in a TCAM (ternary content-addressable memory, which compares every entry in parallel in one clock cycle). Software routers like the Linux kernel use a compressed trie (LC-trie) to look up a full internet table of about 1 million IPv4 prefixes in tens of nanoseconds.

LPM is also how traffic engineering works. Announce a more specific prefix and you attract traffic. That's the mechanism behind both anycast and BGP hijacks (BGP).


Routing Algorithms

Routing protocols compute least-cost paths over a graph: routers are nodes, links are edges with costs. There are two fundamental approaches.

Distance vectorLink state
What a router knowsOnly its neighbours' distances to each destinationThe entire topology (every link and cost)
What it sendsIts distance vector, to neighbours onlyIts own links, flooded to every router
AlgorithmDistributed Bellman-FordLocal Dijkstra on the full graph
ConvergenceSlow, iterative, can count to infinityFast, everyone computes from the same map
CPU and memoryLowHigher (LSDB + SPF runs)
ExamplesRIP, EIGRP (advanced DV), BabelOSPF, IS-IS
RelatedBGP is a path vector protocol, a DV variant that sends full AS paths to prevent loops

Distance Vector: Distributed Bellman-Ford

Each router x keeps an estimate Dₓ(y) of its cost to every destination y and repeatedly applies the Bellman-Ford equation:

"My best cost to y is the best over my neighbours of (cost to that neighbour + that neighbour's advertised cost to y)." Whenever its vector changes, a router sends it to its neighbours. Real protocols also send it periodically.

Worked Example: Convergence

Five routers:

flowchart LR
    A((A)) ---|"1"| B((B))
    A ---|"5"| C((C))
    B ---|"2"| C
    B ---|"4"| E((E))
    C ---|"1"| D((D))
    D ---|"1"| E

Round 0: each router knows only its direct links.

From \ ToABCDE
A015∞∞
B102∞4
C5201∞
D∞∞101
E∞4∞10

Round 1: every router applies Bellman-Ford to its neighbours' round-0 vectors. Here's A's computation (neighbours B at cost 1 and C at cost 5):

A tovia B: 1 + D_Bvia C: 5 + D_CBestNext hop
B1 + 0 = 15 + 2 = 71B
C1 + 2 = 35 + 0 = 53B
D1 + ∞ = ∞5 + 1 = 66C
E1 + 4 = 55 + ∞ = ∞5B

Full table after round 1:

From \ ToABCDE
A01365
B10234
C32012
D63101
E54210

Round 2: A now hears B's improved D_B(D) = 3, so D_A(D) = min(1 + 3, 5 + 1) = 4 via B (was 6 via C). Likewise D learns D_D(A) = min(1 + D_C(A), 1 + D_E(A)) = min(1 + 3, 1 + 5) = 4 via C.

From \ ToABCDE
A01345
B10234
C32012
D43101
E54210

Round 3: no entry changes, so the network has converged after 2 rounds of exchanges. A's final forwarding table sends everything via B: B (1), C (3), D (4), E (5). Note that the direct A–C link (cost 5) is never used, because A→B→C costs 3.

Good news travels fast: a newly better path spreads in about one round per hop.

The Count-to-Infinity Problem

Bad news travels slowly. Take a chain:

A ──1── B ──1── C

Converged: D_B(C) = 1 (direct), D_A(C) = 2 (via B). Now the B–C link fails.

B no longer has a direct route to C, but A is still advertising "I can reach C at cost 2." B doesn't know that A's route goes through B, so it believes A:

ExchangeB's cost to CA's cost to CWhat happened
before failure1 (direct)2 (via B)—
13 (via A: 1 + 2)2B believes A's stale route
234 (via B: 1 + 3)A updates from B's new vector
354…
456…
………packets for C loop between A and B the whole time
131514…
141516 = ∞A reaches RIP's "infinity"
1516 = ∞16C finally declared unreachable by both

With RIP's 30-second update interval, this takes minutes, and the packets loop until TTL expires. That's why RIP caps the maximum at 15 hops (16 = infinity). A larger "infinity" would just take longer to count to.

Fixes

  • Split horizon: don't advertise a route back out the interface you learned it on. A learned C from B, so A doesn't tell B about C at all. When B–C fails, B has no false alternative and immediately marks C unreachable.
  • Split horizon with poison reverse: go further and advertise the route back to where you learned it with metric ∞. A tells B "C = 16". This actively kills any stale belief instead of waiting for timeouts.
  • Triggered updates: send changes immediately instead of waiting for the next periodic update.
  • Hold-down timers: after a route goes unreachable, ignore worse offers for it for a while.

Split horizon doesn't fix every loop

Split horizon and poison reverse prevent 2-node loops. With three or more routers in a cycle (A tells B, B tells C, C tells A), a stale route can still circulate and count to infinity. The real fixes are protocols that carry more than a number: EIGRP's feasibility condition (DUAL), Babel's sequence numbers, or BGP's full AS path, which lets a router reject any path containing itself.

Each router:

  1. Discovers neighbours with Hello packets.
  2. Builds a Link-State Advertisement (LSA): "I am R5, my links are R2 cost 3 and R7 cost 1." Each LSA carries a sequence number and an age.
  3. Floods the LSA to every router. Each router forwards an LSA out every interface except the one it came in on, if it's newer than the copy it already has (by sequence number). Old or duplicate copies are dropped, so flooding terminates.
  4. Ends up with an identical Link-State Database (LSDB), a full map of the area.
  5. Runs Dijkstra's shortest-path-first (SPF) algorithm with itself as the root, and installs the results in its forwarding table.

Worked Example: Dijkstra from Router u

flowchart LR
    u((u)) ---|"2"| v((v))
    u ---|"1"| x((x))
    u ---|"5"| w((w))
    v ---|"2"| x
    v ---|"3"| w
    x ---|"3"| w
    x ---|"1"| y((y))
    w ---|"1"| y
    w ---|"5"| z((z))
    y ---|"2"| z

N' is the set of nodes whose shortest path is final. D(n), p(n) are the current best distance and the predecessor on that path. At each step, add the unfinalised node with the smallest D and relax its neighbours.

StepN'D(v), pD(w), pD(x), pD(y), pD(z), p
0u2, u5, u1, u∞∞
1u x2, u4, x—2, x∞
2u x v—4, x—2, x∞
3u x v y—3, y——4, y
4u x v y w————4, y
5u x v y w z—————

Reading it step by step:

  • Step 1, add x (1). Via x: w = 1 + 3 = 4 < 5, so update. y = 1 + 1 = 2. v = 1 + 2 = 3, which isn't better than 2.
  • Step 2, v and y are tied at 2. Pick v (alphabetical tie-break). Via v: w = 2 + 3 = 5, not better than 4.
  • Step 3, add y (2). Via y: w = 2 + 1 = 3 < 4, update. z = 2 + 2 = 4.
  • Step 4, add w (3). Via w: z = 3 + 5 = 8, not better than 4.
  • Step 5, add z (4). Done.

Shortest-path tree and u's forwarding table, found by walking predecessors back to u's first hop:

DestinationCostPathNext hop
v2u → vv
x1u → xx
y2u → x → yx
w3u → x → y → wx
z4u → x → y → zx

Complexity: O(N²) with a simple array, O((E + N) log N) with a binary heap. Fast enough that an OSPF area of hundreds of routers reconverges in well under a second, once flooding finishes and SPF throttle timers allow.

Why every router computes the same answer

Every router in an area has the same LSDB and runs the same deterministic algorithm, so their trees are consistent and forwarding is loop-free once flooding completes. The transient window, while some routers have the new LSA and some don't, is when microloops happen. Techniques like LFA/TI-LFA fast reroute pre-compute backup next hops to cover it.


Routing Protocols in Practice

flowchart TB
    subgraph AS1["Autonomous System 65001 (your company)"]
        direction LR
        subgraph A0["OSPF Area 0 (backbone)"]
            ABR1["ABR 1"] --- CORE["Core router"] --- ABR2["ABR 2"]
        end
        subgraph A1["Area 1"]
            R1["Router"] --- R2["Router"]
        end
        subgraph A2["Area 2"]
            R3["Router"] --- R4["Router"]
        end
        R1 --- ABR1
        R3 --- ABR2
        CORE --- ASBR["ASBR / border router"]
    end
    ASBR ===|"eBGP"| ISP["ISP AS 64500"]

Interior gateway protocols (IGPs): RIP, OSPF and IS-IS route within an autonomous system, optimising a technical metric. The exterior gateway protocol BGP routes between ASes, applying business policy.

RIP (Routing Information Protocol)

PropertyValue
TypeDistance vector
MetricHop count. Every link costs 1, regardless of bandwidth
Infinity16, so the network diameter is limited to 15 hops
UpdatesFull table every 30 s over UDP port 520 (RIPv2 multicast to 224.0.0.9)
TimersRoute invalid after 180 s without updates, flushed after 240 s, hold-down 180 s
Loop preventionSplit horizon, poison reverse, triggered updates, hold-down
VersionsRIPv1 classful with broadcast. RIPv2 adds CIDR masks, multicast and authentication. RIPng for IPv6

It's simple and still found on small networks and in textbooks. Its hop-count metric means it happily prefers two 10 Mbit/s hops over three 10 Gbit/s ones.

OSPF (Open Shortest Path First)

PropertyValue
TypeLink state (Dijkstra)
TransportDirectly over IP, protocol 89. Multicast to 224.0.0.5 (all OSPF routers) and 224.0.0.6 (DR/BDR)
MetricCost = reference bandwidth / interface bandwidth. The reference defaults to 100 Mbit/s
HellosEvery 10 s, dead interval 40 s on broadcast networks (often tuned down, or replaced by BFD for sub-second failure detection)
HierarchyAreas. Area 0 is the backbone, and every other area must attach to it
VersionsOSPFv2 (IPv4), OSPFv3 (IPv6, and IPv4 via address families)

Key concepts:

  • Areas contain flooding. LSAs about an area's internal topology stay inside the area. Area Border Routers (ABRs) summarise the area into the backbone as prefix lists, not topology. A link flapping in Area 1 triggers SPF only within Area 1. Autonomous System Boundary Routers (ASBRs) inject external routes (from BGP or static).
  • Designated Router (DR) and Backup DR. On a broadcast segment with n routers, full adjacency between all pairs would mean n(n − 1)/2 relationships flooding LSAs at each other. Instead, everyone adjoins only the DR and BDR, which are elected by priority and then router ID.
  • Neighbour states: Down → Init → 2-Way → ExStart → Exchange → Loading → Full. An adjacency stuck in ExStart or Exchange is almost always an MTU mismatch between the two ends.
  • Area types: stub and totally stubby areas (which block external and inter-area LSAs, replacing them with a default route), and NSSA.

The reference bandwidth trap

With the default 100 Mbit/s reference, a 100 Mbit/s link and a 100 Gbit/s link both get cost 1, because cost can't go below 1. OSPF then treats them as equal. Set auto-cost reference-bandwidth to at least your fastest link speed (for example 100000 Mbit/s), identically on every router.

IS-IS (Intermediate System to Intermediate System)

Another link-state protocol with Dijkstra and flooding, originally designed for OSI's CLNP. Differences from OSPF:

  • It runs directly over layer 2, not IP, so it can't be attacked or spoofed with IP packets from afar, and it's protocol-agnostic. IPv4 and IPv6 are just TLVs (type-length-value fields) in the same LSPs.
  • Two-level hierarchy (Level 1 within an area, Level 2 backbone) with area boundaries on links rather than inside routers.
  • The TLV encoding makes it easy to extend (segment routing, traffic engineering).

It's the IGP of choice for many large ISPs and hyperscale data centres.

BGP at a Glance

BGP is a path vector protocol: routers advertise prefixes along with the full AS path to reach them. A router rejects any path that already contains its own AS number, which prevents loops without counting to infinity. Route choice is driven by policy (local preference, AS path length, MED, communities) rather than a shortest-path metric. It runs over TCP port 179. It's how roughly 75,000 autonomous systems exchange about 1 million IPv4 and 200,000 IPv6 prefixes. Full detail, including hijacks and RPKI, is in BGP.

RIPOSPFIS-ISBGP
ScopeIGP, smallIGP, enterprise or campusIGP, ISP or data centreEGP, inter-AS (also data centre fabrics)
AlgorithmDistance vectorLink stateLink statePath vector
MetricHopsBandwidth-based costConfigurable costPolicy attributes
TransportUDP 520IP proto 89Layer 2 directlyTCP 179
ConvergenceMinutesSub-second to secondsSub-second to secondsSeconds to minutes
Scale≤ 15 hopsHundreds of routers per areaThousandsThe whole internet

ICMP: The Network's Error and Diagnostic Channel

The Internet Control Message Protocol (IP protocol 1) carries errors and diagnostics about IP packets. An ICMP error includes the IP header plus at least the first 8 bytes of the packet that caused it. That's enough for the sender to match it back to a socket by ports.

TypeNameCodes / use
0Echo ReplyResponse to ping
3Destination Unreachable0 net, 1 host, 3 port (UDP to a closed port), 4 fragmentation needed and DF set (PMTUD), 13 administratively prohibited (firewall)
5Redirect"Use a better gateway on your own subnet." Usually ignored by hosts for security
8Echo Requestping
11Time Exceeded0 TTL expired in transit (traceroute), 1 fragment reassembly timeout
12Parameter ProblemMalformed header

Rules that keep ICMP from making things worse: no ICMP errors about ICMP errors, none about non-first fragments, none about broadcast or multicast packets. Routers also rate-limit ICMP generation (net.ipv4.icmp_ratelimit on Linux), which is why traceroute sometimes shows * * * at a hop that's actually forwarding fine.

ping

ping -c 4 1.1.1.1
# 64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=11.8 ms
# ...
# 4 packets transmitted, 4 received, 0% packet loss, time 3005ms
# rtt min/avg/max/mdev = 11.512/11.790/12.104/0.221 ms
  • ttl=57 is the TTL left on the reply. Common initial values are 64 (Linux, macOS), 128 (Windows) and 255 (many routers). The reply most likely started at 64, so it crossed about 7 hops.
  • mdev is jitter. High mdev with low loss usually means queueing (bufferbloat) somewhere on the path.
  • ICMP may be deprioritised or blocked. "Ping fails" does not prove "host is down." Try nc -vz host 443 or curl for the real service.

traceroute: TTL as a Probe

traceroute sends probes with TTL = 1, 2, 3, …. Each router that decrements a probe's TTL to 0 drops it and returns ICMP Time Exceeded, revealing its address. The destination itself answers differently. Classic Unix traceroute sends UDP to unlikely ports (33434 and up), so the destination replies ICMP Port Unreachable, which signals the end.

sequenceDiagram
    participant H as Host
    participant R1 as Router 1
    participant R2 as Router 2
    participant D as Destination
    H->>R1: UDP probe, TTL 1, port 33434
    R1->>H: ICMP Time Exceeded from R1
    H->>R1: UDP probe, TTL 2, port 33435
    R1->>R2: forwarded with TTL 1
    R2->>H: ICMP Time Exceeded from R2
    H->>R1: UDP probe, TTL 3, port 33436
    R1->>R2: TTL 2
    R2->>D: TTL 1
    D->>H: ICMP Port Unreachable, destination reached

By default it sends 3 probes per TTL, giving the three RTT columns.

traceroute -n example.com          # UDP probes (Linux default), -n skips reverse DNS
traceroute -n -I example.com       # ICMP echo probes (what Windows tracert uses)
sudo traceroute -n -T -p 443 example.com   # TCP SYN probes to 443, which gets through most firewalls
mtr -rwzc 100 example.com          # traceroute + ping combined: per-hop loss and latency over 100 cycles

Reading traceroute correctly:

  • * * * at one hop but later hops respond: that router just doesn't send or rate-limits ICMP. It's not packet loss.
  • Loss that starts at hop N and persists to the destination is real loss at or after hop N. Loss at hop N only, with clean hops after it, is ICMP deprioritisation on that router's CPU.
  • Latency that jumps and stays high marks a long link (a submarine cable, for example). A single high hop followed by lower ones is again slow ICMP generation.
  • Paths are asymmetric. traceroute shows only the forward path. The replies may come back a completely different way.
  • ECMP: load-balanced paths can make different probes take different routes, which shows several IPs per hop. Paris traceroute and mtr fix the flow hash to keep probes on one path.

More on this in Network Debugging.


DHCP: Getting an Address

A host joining a network needs at least an IP address, a subnet mask, a default gateway and DNS servers. The Dynamic Host Configuration Protocol (RFC 2131) hands them out from a pool. It runs over UDP, with the server on port 67 and the client on port 68.

The DORA Exchange

sequenceDiagram
    participant C as Client no IP yet
    participant S as DHCP Server 10.0.0.2
    C->>S: DISCOVER from 0.0.0.0 to 255.255.255.255, xid 0x3903F326
    S->>C: OFFER yiaddr 10.0.0.57, mask, router, DNS, lease 86400 s
    C->>S: REQUEST broadcast, requested IP 10.0.0.57, server id 10.0.0.2
    Note over S: other servers that offered see the REQUEST and withdraw
    S->>C: ACK 10.0.0.57 is yours for 24 h
    Note over C: optional ARP probe for 10.0.0.57, send DECLINE if someone answers
    Note over C: configure interface, start T1 and T2 timers
StepMessageSrc → DstKey content
DDHCPDISCOVER0.0.0.0:68 → 255.255.255.255:67Client MAC (chaddr), transaction ID xid, parameter request list
ODHCPOFFERserver:67 → broadcast or client:68yiaddr (your IP), option 1 mask, 3 router, 6 DNS, 51 lease time, 54 server ID
RDHCPREQUEST0.0.0.0:68 → 255.255.255.255:67Option 50 requested IP, option 54 the chosen server. Broadcast so rejected servers know
ADHCPACKserver:67 → client:68Final confirmation and parameters

Other messages: NAK (your requested address isn't valid here, for example after moving networks), DECLINE (the offered address is already in use, detected by ARP), RELEASE (giving it back), INFORM (I have an address, just send me the options).

Why the REQUEST is broadcast: several servers may have made offers, and the broadcast tells all of them which one was accepted, so the others can return their offered addresses to the pool.

Leases and Renewal

A lease is temporary. The client renews it in the background:

TimerDefaultAction
T1 (renewal)50% of the leaseUnicast REQUEST to the original server
T2 (rebinding)87.5% of the leaseThe original server isn't answering, so broadcast REQUEST to any server
Expiry100%Stop using the address and restart from DISCOVER

Worked example. A 24-hour lease is granted at 08:00.

  • T1 = 08:00 + 12 h = 20:00: unicast renew. If ACKed, the lease restarts from 20:00.
  • If the server is down: T2 = 08:00 + 21 h = 05:00 next day: broadcast rebind.
  • If nothing answers: expiry at 08:00 next day, the interface loses its address, and existing connections break.

Short leases (minutes) suit busy guest Wi-Fi with fast address recycling. Long leases (days) suit stable offices and survive DHCP server outages.

DHCP Relay

DISCOVER is a broadcast, and broadcasts don't cross routers. A network with 50 VLANs doesn't run 50 DHCP servers. Instead, each VLAN's router interface runs a relay agent (ip helper-address on Cisco):

  1. The relay receives the client's broadcast on VLAN 20.
  2. It sets giaddr (gateway IP address) to its own interface address in VLAN 20, for example 10.20.0.1, and unicasts the message to the central server.
  3. The server picks a pool by giaddr, so this client gets an address from 10.20.0.0/24, and replies to the relay.
  4. The relay delivers the reply onto VLAN 20.

Option 82 (relay agent information) can add the switch and port the request came in on, for per-port policy or location tracking.

DHCP Security

  • Rogue DHCP server: anyone on the segment can answer DISCOVERs faster and hand out themselves as the gateway or DNS server, a perfect man-in-the-middle position. The defence is DHCP snooping: the switch only allows OFFER/ACK from trusted ports.
  • Starvation: flooding DISCOVERs from random MACs exhausts the pool. Defences are port security and DHCP snooping rate limits.
  • The snooping binding table also feeds Dynamic ARP Inspection and IP source guard (see The Data Link Layer).

DHCP on Linux

sudo dhclient -v eth0          # run DORA verbosely (ISC dhclient)
# DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 3
# DHCPOFFER of 10.0.0.57 from 10.0.0.2
# DHCPREQUEST for 10.0.0.57 on eth0 to 255.255.255.255 port 67
# DHCPACK of 10.0.0.57 from 10.0.0.2
# bound to 10.0.0.57 -- renewal in 41234 seconds.

sudo dhclient -r eth0          # release the lease
cat /var/lib/dhcp/dhclient.leases            # lease details (path varies by distro)
networkctl status eth0         # systemd-networkd's view, including DHCP state
nmcli device show eth0         # NetworkManager's view

# Watch the exchange on the wire
sudo tcpdump -i eth0 -n -v 'port 67 or port 68'

Many modern distributions use systemd-networkd or NetworkManager's internal client instead of dhclient, but the protocol is identical.


IPv6

IPv4's 32 bits allow 4.3 billion addresses. They ran out at IANA in 2011, and the internet kept working only through NAT and CGNAT. IPv6 has 128-bit addresses, about 3.4 × 10³⁸. It also fixes several IPv4 design mistakes. Google measures roughly half of its users reaching it over IPv6, and mobile networks are often IPv6-first.

Address Format and Shorthand

Eight groups ("hextets") of four hex digits, separated by colons:

2001:0db8:0000:0000:0000:ff00:0042:8329

Compression rules (RFC 5952 canonical form):

  1. Drop leading zeros in each group: 0db8 → db8, 0042 → 42, 0000 → 0.
  2. Replace the longest run of all-zero groups with ::, once only. If two runs tie, compress the leftmost. Don't use :: for a single zero group.
  3. Use lowercase hex.

Worked examples:

Full formStep 1 (drop leading zeros)Canonical
2001:0db8:0000:0000:0000:ff00:0042:83292001:db8:0:0:0:ff00:42:83292001:db8::ff00:42:8329
fe80:0000:0000:0000:0000:0000:0000:0001fe80:0:0:0:0:0:0:1fe80::1
2001:0db8:0000:0001:0000:0000:0000:00012001:db8:0:1:0:0:0:12001:db8:0:1::1 (compress the run of three, not the single zero)
0000:0000:0000:0000:0000:0000:0000:00010:0:0:0:0:0:0:1::1 (loopback)

Expanding: 2001:db8::ff00:42 has 4 groups written, so :: stands for 8 − 4 = 4 zero groups: 2001:0db8:0000:0000:0000:0000:ff00:0042.

2001::db8::1 is invalid, because with two :: you can't tell how many zeros each one stands for.

In URLs, IPv6 literals need brackets: https://[2001:db8::1]:8443/.

Address Types

IPv6 has no broadcast. Its functions are taken over by multicast.

PrefixTypeNotes
2000::/3Global unicastInternet-routable. Typically a site gets a /48 or /56, and each subnet is a /64
fe80::/10Link-localAuto-configured on every interface and never routed. Used by NDP and routing protocols. Needs a zone: fe80::1%eth0
fc00::/7 (in practice fd00::/8)Unique local (ULA)The equivalent of private RFC 1918 space
ff00::/8Multicastff02::1 all nodes, ff02::2 all routers, ff02::1:ffXX:XXXX solicited-node
::1/128Loopback
::/128UnspecifiedSource address before you have one
::ffff:0:0/96IPv4-mapped::ffff:192.0.2.1, how dual-stack sockets represent IPv4 peers
64:ff9b::/96NAT64 well-known prefixIPv4 addresses embedded for translation

An IPv6 interface normally has several addresses at once: a link-local, one or more global addresses (stable and temporary privacy ones), and maybe a ULA. Source address selection (RFC 6724) picks which to use.

The /64 and Interface Identifiers

A global unicast address is 64-bit network prefix + 64-bit interface ID. The interface ID can be:

  • Modified EUI-64, derived from the MAC. Now discouraged, because it makes the device trackable across networks.
  • Random temporary addresses (RFC 8981 privacy extensions), rotated daily. These are the default for outgoing connections on most OSes.
  • Stable-privacy (RFC 7217): a hash of the prefix, interface and secret. Stable per network, different across networks.

Worked example: EUI-64 from MAC 00:1a:2b:3c:4d:5e.

  1. Split the MAC in half and insert ff:fe: 00:1a:2b ff:fe 3c:4d:5e.
  2. Flip the U/L bit (bit 1 of the first byte): 0x00 = 00000000 → 00000010 = 0x02.
  3. Interface ID = 021a:2bff:fe3c:4d5e.
  4. Link-local address: fe80::21a:2bff:fe3c:4d5e. With prefix 2001:db8:1:2::/64: 2001:db8:1:2:21a:2bff:fe3c:4d5e.

The IPv6 Header

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Version| Traffic Class |              Flow Label               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|         Payload Length        |  Next Header  |   Hop Limit   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                    Source Address (128 bits)                  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                 Destination Address (128 bits)                |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
IPv4 fieldIPv6 equivalentWhy it changed
IHL, optionsGone. Fixed 40-byte header, options moved into extension headers chained via Next HeaderFixed size makes hardware parsing simple
Total LengthPayload Length (excludes the 40-byte header)
TOSTraffic ClassSame DSCP/ECN meaning
—Flow Label (20 bits)Lets routers hash flows for ECMP without digging into L4
ID, flags, fragment offsetGone from the base header. A Fragment extension header, used by the source onlyRouters never fragment, so PMTUD is mandatory
TTLHop LimitRenamed to say what it actually counts
ProtocolNext HeaderPoints at the next extension header or the L4 protocol (6 TCP, 17 UDP, 58 ICMPv6)
Header checksumGoneL2 has CRC and L4 has checksums, so routers skip a per-hop recompute. The UDP checksum becomes mandatory

Extension headers: Hop-by-Hop Options, Routing (used by SRv6 segment routing), Fragment, Destination Options, AH, ESP. In practice, packets with unusual extension headers are often dropped on the public internet.

NDP: Neighbour Discovery Replaces ARP

IPv6 has no ARP. Neighbour Discovery Protocol (RFC 4861) runs as ICMPv6 messages and does much more:

ICMPv6 typeMessageReplaces / does
133Router Solicitation (RS)"Any routers here?" sent at interface startup
134Router Advertisement (RA)Prefix, default gateway, MTU, flags (M/O bits say whether to use DHCPv6). Replaces part of DHCP and ICMP router discovery
135Neighbour Solicitation (NS)ARP request equivalent, also used for duplicate address detection
136Neighbour Advertisement (NA)ARP reply and gratuitous ARP equivalent
137RedirectICMP redirect equivalent

Address resolution doesn't broadcast. The NS goes to the target's solicited-node multicast address, ff02::1:ff + the low 24 bits of the target address. For 2001:db8::21a:2bff:fe3c:4d5e that's ff02::1:ff3c:4d5e. NICs filter multicast in hardware, so only the one or two hosts sharing those 24 bits even look at the packet. With ARP, every host on the segment processes every request.

NDP is as unauthenticated as ARP. RA guard on switches (dropping RAs from host ports) is the IPv6 equivalent of DHCP snooping, and SEND (Secure Neighbour Discovery) exists but is rarely deployed.

SLAAC: Stateless Address Autoconfiguration

A host can configure a global address without any DHCP server:

sequenceDiagram
    participant H as Host
    participant N as Link, all nodes
    participant R as Router
    Note over H: build link-local fe80 address from an interface ID
    H->>N: NS for own tentative address, DAD
    Note over H: no NA received within about 1 s, address is unique
    H->>R: Router Solicitation to the all-routers multicast group
    R->>H: Router Advertisement with one /64 prefix, A flag set, router lifetime 1800 s
    Note over H: prefix plus interface ID gives a global address
    H->>N: DAD again for the new global address
    Note over H: default route via the router's link-local address
  • DAD (Duplicate Address Detection): before using an address, send an NS for it from ::. Any answer means it's taken.
  • The default gateway is the router's link-local address (ip -6 route shows default via fe80::1 dev eth0). That way it survives prefix renumbering.
  • DNS servers come from the RA's RDNSS option (RFC 8106), or from stateless DHCPv6 (O flag).
  • Stateful DHCPv6 (M flag) assigns addresses like IPv4 DHCP, for enterprises that want address logging. Android notably doesn't support it, so most networks run SLAAC.

Dual Stack and Transition

The internet couldn't switch overnight, so several mechanisms coexist:

MechanismHow it worksWhere it's used
Dual stackEvery host and router runs both IPv4 and IPv6The default target. Most clouds, ISPs, OSes
Happy Eyeballs (RFC 8305)The client resolves A and AAAA, tries IPv6 first, and races IPv4 after about 250 ms if IPv6 is slowEvery major browser and OS. Hides broken IPv6 from users
6in4 / GRE tunnelsIPv6 packets inside IPv4 (protocol 41) across IPv4-only networksTunnel brokers, legacy links
6to4, TeredoAutomatic tunnellingDeprecated, unreliable
NAT64 + DNS64IPv6-only clients reach IPv4 servers. DNS64 synthesises AAAA records inside 64:ff9b::/96, and NAT64 translatesIPv6-only mobile networks, AWS NAT gateway with DNS64
464XLATCLAT on the device translates IPv4 apps to IPv6, and the provider's NAT64 translates backT-Mobile and many mobile carriers, Android
DS-LiteIPv4 tunnelled over an IPv6-only access network to a carrier NATSome cable/fibre ISPs

Address ranges and dual-stack VPC design are also covered in IP Addressing & Subnetting. IPv6 makes NAT unnecessary for addressing, but stateful firewalls still matter. "Globally routable" doesn't mean "reachable": the default should still be to deny inbound.

ip -6 addr show dev eth0
#   inet6 2001:db8:1:2:9c4e:1fd2:a3b0:7e11/64 scope global temporary dynamic
#   inet6 2001:db8:1:2:21a:2bff:fe3c:4d5e/64 scope global dynamic mngtmpaddr
#   inet6 fe80::21a:2bff:fe3c:4d5e/64 scope link
ip -6 route                     # default via fe80::1 dev eth0 proto ra metric 100
ip -6 neigh                     # the NDP cache (IPv6's ARP table)
ping -6 -c 2 ff02::1%eth0       # every IPv6 node on the link replies
rdisc6 eth0                     # send an RS and print the RA (ndisc6 package)
sudo tcpdump -i eth0 -n 'icmp6 and (ip6[40] >= 133 and ip6[40] <= 137)'   # watch NDP

Routers vs Layer 3 Switches

RouterLayer 3 switch
Core jobRoute between diverse networks, WAN edgeHigh-speed routing between VLANs inside a campus or data centre
ForwardingASIC or network processor, sometimes softwareSwitching ASIC with routing in hardware (TCAM)
PortsMixed: Ethernet, fibre WAN, cellular, legacy serialDense Ethernet: 48 × 25G, 32 × 400G
FeaturesFull BGP tables, deep buffers, NAT, VPN, QoS shaping, firewallInter-VLAN routing via SVIs (switch virtual interfaces), OSPF/BGP, limited NAT and VPN
Price per portHighLow
Typical placeInternet edge, branch WANCampus distribution, data centre leaf/spine

The line has blurred. Modern data centre "switches" run full BGP on merchant silicon, which is the basis of leaf-spine fabrics where every link is a routed L3 link with ECMP and no spanning tree. The remaining distinctions are table scale (a full internet table needs a lot of FIB memory), buffer depth, and WAN features.

Router-on-a-stick is the older alternative to an L3 switch: one router interface with 802.1Q sub-interfaces (one per VLAN) on a trunk to the switch, routing between VLANs over a single cable.


Routing on a Linux Box

Linux is a full router. Every host has a routing table, and setting one sysctl turns it into a forwarder.

ip route show
# default via 192.168.1.1 dev eth0 proto dhcp src 192.168.1.50 metric 100
# 10.8.0.0/24 dev tun0 proto kernel scope link src 10.8.0.2
# 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
# 192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.50 metric 100
PartMeaning
default0.0.0.0/0, the route of last resort
via 192.168.1.1Next-hop gateway, ARPed for its MAC
no via + scope linkOn-link: the destination is ARPed directly
proto kernel / dhcp / static / birdWho installed it: connected-route automation, the DHCP client, an admin, a routing daemon
srcPreferred source address for locally originated packets
metricTie-breaker between routes to the same prefix. Lower wins
linkdownThe interface has no carrier, so the route is unusable
# Which route would be used? (LPM, policy rules and all) — the single most useful routing command
ip route get 8.8.8.8
# 8.8.8.8 via 192.168.1.1 dev eth0 src 192.168.1.50 uid 1000
#     cache

ip route get 10.8.0.7             # 10.8.0.7 dev tun0 src 10.8.0.2   (on-link via the VPN)

# Add and remove routes
sudo ip route add 10.20.0.0/16 via 192.168.1.254 dev eth0
sudo ip route replace default via 192.168.1.1 dev eth0 metric 50
sudo ip route del 10.20.0.0/16
sudo ip route add blackhole 203.0.113.0/24        # silently drop a prefix

# Turn the host into a router
sudo sysctl -w net.ipv4.ip_forward=1
sudo sysctl -w net.ipv6.conf.all.forwarding=1

# Multiple tables and policy routing: rules choose a table, then LPM runs inside it
ip rule show
# 0:      from all lookup local
# 32766:  from all lookup main
# 32767:  from all lookup default
sudo ip route add default via 10.99.0.1 table 100
sudo ip rule add from 10.99.0.0/24 lookup 100     # traffic sourced from this subnet uses the second uplink

ip route show table local          # local/broadcast addresses the kernel owns

Asymmetric routing and rp_filter

If a packet arrives on an interface that isn't the one the kernel would use to reply to its source, reverse path filtering (net.ipv4.conf.*.rp_filter=1, strict mode) drops it silently. Multi-homed hosts, policy routing and some VPN setups hit this. Use loose mode (2) or fix the routing so it's symmetric. The packets show up in tcpdump but never reach the socket, which is the giveaway.

In a network namespace or container, the same commands show the container's own table, typically a default route to the bridge or veth peer (see Namespaces & cgroups). Kubernetes CNIs either program routes per node pod CIDR (Calico in BGP mode, Cilium native routing) or encapsulate (VXLAN). See Kubernetes CNI Networking. Where the kernel makes the routing decision in its receive path is covered in The Linux Networking Stack.


How It Maps to Cloud VPC Route Tables

A VPC is a software-defined layer 3 network, and its route tables are ordinary forwarding tables with longest-prefix match. The targets are cloud constructs instead of next-hop IPs.

A typical AWS public subnet route table:

DestinationTargetMeaning
10.0.0.0/16localEverything inside the VPC. Always present, can't be deleted
10.1.0.0/16pcx-0a1b…A peered VPC
172.16.0.0/12tgw-04c2…On-prem and other VPCs via Transit Gateway (often learned via BGP from Direct Connect or VPN)
0.0.0.0/0igw-08f3…Internet gateway. This is what makes the subnet "public"
::/0igw-08f3…IPv6 internet

A private subnet points 0.0.0.0/0 at a NAT gateway (nat-…) instead, and ::/0 at an egress-only internet gateway (eigw-…), the IPv6 counterpart that allows outbound connections only.

How this maps to classical concepts:

ClassicalCloud VPC
Router and its FIBThe implicit VPC router (at subnet base + 1) and route tables per subnet
Longest-prefix matchSame: the /16 to the peering beats the /0 default
Static routesRoute table entries
Dynamic routing (BGP)Route propagation from VGW/TGW/Direct Connect, Cloud Router (GCP), Azure Route Server
DHCP serverBuilt in, with a DHCP options set for DNS servers, domain name and NTP. The address is fixed to the instance
ICMPWorks, subject to security groups and NACLs. Allow ICMP fragmentation-needed or PMTUD breaks for the 9001 → 1500 path
Policy routing / middleboxesRoutes pointing at an ENI or a Gateway Load Balancer endpoint, more-specific routes within the VPC CIDR to insert a firewall
IPv6 SLAACNot used. The cloud assigns the /64 per subnet (and /56 per VPC on AWS) and hands out addresses via DHCPv6

AWS reserves five addresses per subnet: network, base+1 (VPC router), base+2 (DNS resolver), base+3 (reserved) and the last address. A /28 therefore offers 11 usable hosts, not 14. GCP VPCs are global (subnets are regional, but one route table spans regions), while AWS and Azure VPCs are regional. For details see AWS VPC Networking and, for multi-region routing, Traffic Routing.


Practice Problems

1. Fragmentation. A 2400-byte IPv4 datagram (20-byte header, no options) must cross a link with MTU 820. List each fragment's data size, offset field, MF flag and total length.

Solution

Data = 2400 − 20 = 2380 bytes. Max data per fragment = ⌊(820 − 20)/8⌋ × 8 = 800 (800/8 = 100, exact).

FragmentDataOffset (÷8)MFTotal length
180001820
28001001820
37802000800

Check: 800 + 800 + 780 = 2380 ✔. Bytes on the wire: 2440, 60 more than the original, because of the extra headers.

2. Longest-prefix match. Forwarding table:

PrefixInterface
192.168.0.0/16eth0
192.168.32.0/19eth1
192.168.40.0/21eth2
192.168.44.0/22eth3
0.0.0.0/0eth4

Where do these go: (a) 192.168.45.10 (b) 192.168.41.1 (c) 192.168.60.5 (d) 192.168.100.1 (e) 10.0.0.1?

Solution

Ranges: /19 = 192.168.32.0–63.255. /21 = 192.168.40.0–47.255. /22 = 192.168.44.0–47.255.

  • (a) 192.168.45.10 → in the /16, /19, /21 and /22, so the longest is eth3
  • (b) 192.168.41.1 → in the /16, /19 and /21 (41 < 44, so not the /22) → eth2
  • (c) 192.168.60.5 → in the /16 and /19 (60 > 47) → eth1
  • (d) 192.168.100.1 → only the /16 → eth0
  • (e) 10.0.0.1 → only the default → eth4

3. Distance vector update. Router X has neighbours A (link cost 2), B (cost 5) and C (cost 1). It receives these vectors:

to Ato Bto Cto Dto E
from A03469
from B30214
from C42079

Compute X's new distance and next hop for every destination.

Solution
Destvia A (2 + D_A)via B (5 + D_B)via C (1 + D_C)Best
A2852 via A
B5533 via C
C6711 via C
D8686 via B
E119109 via B

Note that X reaches B more cheaply through C (1 + 2 = 3) than over its direct link (5).

4. Dijkstra. Graph: S–A 4, S–B 1, B–A 2, A–C 1, B–C 5, C–D 3, B–D 8. Run Dijkstra from S and give S's forwarding table.

Solution
StepN'D(A), pD(B), pD(C), pD(D), p
0S4, S1, S∞∞
1S B3, B—6, B9, B
2S B A——4, A9, B
3S B A C———7, C
4S B A C D————

Paths: B = 1 (S→B), A = 3 (S→B→A), C = 4 (S→B→A→C), D = 7 (S→B→A→C→D). Every destination's next hop is B. The direct S–A link (4) loses to S→B→A (3).

5. Count to infinity. In the chain A–B–C (costs 1), the B–C link fails. Without split horizon, using RIP (infinity = 16, updates every 30 s, assuming one exchange per update period), roughly how long until both routers mark C unreachable? What changes with split horizon?

Solution

Costs rise by 2 for each full round-trip of updates (B: 3, 5, 7, … and A: 4, 6, 8, …). Reaching 16 takes about 14 alternating exchanges, which at one 30-second update each is about 7 minutes of looping traffic. Triggered updates make this faster, but the loop is still there.

With split horizon, A never advertises C back to B (A learned it from B), so when B–C fails, B immediately has no route, sets C = 16, and tells A. Convergence takes one update.

6. IPv6 notation. (a) Compress 2001:0db8:0000:0000:0008:0800:200c:417a. (b) Expand 2001:db8:0:1::1. (c) Is 2001:db8::1::2 valid? (d) Give the solicited-node multicast address for 2001:db8::abcd:1234:5678.

Solution

(a) Drop leading zeros: 2001:db8:0:0:8:800:200c:417a. Compress the zero run: 2001:db8::8:800:200c:417a.

(b) Five groups written, so :: = 3 zero groups: 2001:0db8:0000:0001:0000:0000:0000:0001.

(c) Invalid. Two :: are ambiguous.

(d) Low 24 bits: …34:5678 → ff02::1:ff34:5678.

7. EUI-64. Derive the link-local address for MAC a4:83:e7:12:34:56.

Solution

Insert ff:fe: a4:83:e7:ff:fe:12:34:56. Flip the U/L bit of a4 (10100100 → 10100110 = a6). Interface ID = a683:e7ff:fe12:3456. Link-local: fe80::a683:e7ff:fe12:3456.

8. DHCP timers. A client gets a 6-hour lease at 14:00. When does it first try to renew, to whom, and how? If that server is permanently down, when does it rebind and when does the address expire?

Solution
  • T1 = 50% = 3 h → 17:00: unicast DHCPREQUEST to the original server.
  • T2 = 87.5% = 5 h 15 min → 19:15: broadcast DHCPREQUEST to any server.
  • Expiry → 20:00: the address is dropped and the client restarts at DISCOVER.

9. Reading TTLs. ping replies show ttl=115, and traceroute to the same host shows 14 hops. What is the remote host's likely OS, and is the path symmetric?

Solution

115 is just under 128, so the reply started at 128, which suggests Windows. 128 − 115 = 13 routers decremented it, so the reply crossed about 13 hops (14 including the destination as the last "hop"). That's consistent with the 14-hop forward path, but TTLs can't prove symmetry. Only a traceroute from the remote side can.

10. OSPF costs. Two paths from R1 to R4: Path P goes through two 10 Gbit/s links. Path Q is one 100 Mbit/s link. (a) Which does OSPF choose with the default 100 Mbit/s reference? (b) With a 100 Gbit/s reference?

Solution

(a) Cost = max(1, 100 Mbit/s ÷ bandwidth). 10G → 0.01 → floored to 1, so P = 1 + 1 = 2. 100M → 1, so Q = 1. OSPF chooses Q, the slow link.

(b) Reference 100,000 Mbit/s: 10G → 10, so P = 20. 100M → 1000, so Q = 1000. OSPF chooses P. This is why the reference bandwidth must be raised everywhere.

11. Path MTU. A host on a PPPoE link (MTU 1492) sends TCP to a server on Ethernet (MTU 1500). What MSS should the TCP SYN advertise to avoid fragmentation? What is the largest payload size N for which ping -M do -s N succeeds across the path?

Solution

MSS = MTU − 20 (IPv4) − 20 (TCP) = 1492 − 40 = 1452 bytes.

ping data size = MTU − 20 (IP) − 8 (ICMP) = 1492 − 28 = 1464. ping -M do -s 1464 succeeds and -s 1465 fails with "message too long" (or is silently dropped if ICMP is blocked).


Interview Questions

What's the difference between forwarding and routing? Forwarding is the per-packet data-plane action: look up the destination in the FIB with longest-prefix match, decrement TTL, rewrite the L2 header and send it out an interface, in nanoseconds and in hardware. Routing is the control-plane process that computes what goes into the FIB: routing protocols exchange information, the best routes are chosen into the RIB, and the winners are installed, over seconds.

Explain longest-prefix match and why it exists. Several prefixes can cover the same destination. A default route, an aggregate and a specific subnet might all match. The router uses the most specific one, which lets aggregates summarise large ranges while specific routes carve out exceptions. That's how CIDR keeps the global table manageable, and also how more-specific announcements attract traffic (anycast, hijacks).

How does traceroute work? It sends probes with TTL 1, 2, 3 and so on. Each router that decrements TTL to zero discards the probe and returns ICMP Time Exceeded, revealing itself. The destination answers with ICMP Port Unreachable for UDP probes (or an echo reply or SYN-ACK for ICMP or TCP probes), which ends the trace. * * * usually means a router rate-limits or doesn't send ICMP, not loss, and the path shown is forward-only.

Distance vector vs link state? Distance vector routers only share distance estimates with neighbours and iterate Bellman-Ford. That's simple and cheap, but convergence is slow and they count to infinity after failures. Link state routers flood their local links to everyone, build an identical full map, and each run Dijkstra. Convergence is fast and loop-free once flooding completes, at the cost of more memory and CPU. RIP is DV, OSPF and IS-IS are LS, and BGP is path vector.

What is count-to-infinity, and how do split horizon and poison reverse help? After a failure, a router can believe a neighbour's stale advertisement that actually depends on the failed router, and the two keep incrementing each other's costs while packets loop. Split horizon stops advertising a route back to the neighbour it was learned from. Poison reverse advertises it back with infinite cost. Both prevent two-router loops but not longer cycles, which need path information (BGP) or feasibility conditions (EIGRP).

Why does OSPF use areas? To contain flooding and SPF computation. Topology detail stays inside an area. ABRs summarise prefixes into the backbone (area 0). So a flapping link in one area doesn't make every router in the network re-run SPF, and LSDBs stay small.

Walk through DHCP. DORA: the client broadcasts DISCOVER from 0.0.0.0. Servers OFFER an address with options (mask, gateway, DNS, lease). The client broadcasts REQUEST naming the chosen server, so the others withdraw. The server ACKs. The client renews by unicast at T1 (50%), rebinds by broadcast at T2 (87.5%), and gives up the address at expiry. Across subnets, a relay agent sets giaddr and unicasts to a central server, which picks the pool based on giaddr.

What breaks if you block all ICMP? Path MTU Discovery. Routers can't tell senders that a DF packet is too big, so large packets disappear into a black hole while small ones work: connections open and then hang. In IPv6 it's worse, because NDP (address resolution, router discovery, DAD) is ICMPv6. Block echo requests if policy requires it, but always allow fragmentation-needed and Packet Too Big, Time Exceeded, and the NDP types.

Why did IPv6 remove fragmentation from routers and the header checksum? Router fragmentation costs CPU, amplifies loss and breaks middleboxes, so IPv6 makes the source do PMTUD and fragment if it must. The header checksum had to be recomputed at every hop because of TTL changes, while link layers already have CRCs and transports have their own checksums. Removing it simplifies hardware forwarding. The fixed 40-byte header with extension headers serves the same goal.

How does a host get an IPv6 address without DHCP? SLAAC. It forms a link-local address, runs Duplicate Address Detection with a Neighbour Solicitation, sends a Router Solicitation, and receives a Router Advertisement with a /64 prefix. It combines the prefix with a random or stable-privacy interface ID, runs DAD again, and uses the router's link-local address as its default gateway. DNS comes from the RA's RDNSS option or stateless DHCPv6.

What replaces ARP in IPv6, and why is it better? NDP, specifically Neighbour Solicitation and Advertisement. The solicitation goes to the target's solicited-node multicast address (ff02::1:ff + the low 24 bits), not broadcast, so NIC hardware filters it out on almost every other host. NDP also handles router discovery, prefix information, redirects and duplicate detection in one protocol.

Router vs L3 switch? Both route IP. L3 switches do it in a switching ASIC at line rate across dense Ethernet ports, mainly between VLANs (SVIs) inside campuses and data centres. Routers add WAN interfaces, deep buffers, full internet BGP tables, NAT, VPN and rich QoS. Modern leaf-spine fabrics blur the line by running BGP on switches.

How does a cloud VPC route table relate to a classical router? It's a forwarding table with longest-prefix match applied by the implicit VPC router, with cloud targets instead of next-hop IPs: local for the VPC CIDR, an internet gateway for the public default route, a NAT gateway for private egress, plus peering, transit gateway or ENI targets. BGP propagation from VPN or Direct Connect fills in on-prem routes, and DHCP is built in.

Two Linux interfaces on different subnets, replies leave by the wrong one and connections fail. What's happening and how do you fix it? There's only one default route, in the main table, so replies to traffic that arrived on the second interface go out the first. Upstream drops them as spoofed, or rp_filter drops the inbound side. Fix with policy routing: a second table with its own default via the second gateway, and ip rule add from <second-IP> lookup <table>. Verify with ip route get <dst> from <src>.