Application Layer Protocols: Email, File Transfer, Remote Access & More
The transport layer gives you a pipe. TCP promises that bytes arrive in order and without gaps; UDP promises only that each datagram either arrives whole or not at all. Neither says anything about what the bytes mean. That is the job of the application layer: a set of agreements about who speaks first, what a request looks like, how a reply signals success or failure, and when the conversation is over.
Most engineers only ever write HTTP. But the rest of the stack still runs your infrastructure. Password-reset emails travel over SMTP. Your deploy pipeline pushes artifacts over SFTP or SSH. Every server's clock is disciplined by NTP, and when it drifts your TLS certificates "expire" early and your distributed locks stop being safe. Monitoring systems still poll switches with SNMP. Corporate logins hit LDAP. When one of these breaks, the debugging skill is the same every time: open a raw connection, speak the protocol by hand, and read what comes back.
This page is the CS-fundamentals tour of those protocols. It assumes you know the layer model from OSI Model & TCP/IP Stack and roughly how TCP works from TCP/IP Deep Dive. Protocols that already have their own page here (DNS, HTTP/2 and HTTP/3, TLS, WebSockets, gRPC, MQTT) get a short recap and a link rather than a repeat.
Part 1: How Applications Talk
Client-Server vs Peer-to-Peer
Every application protocol assumes an architecture: who initiates, who holds the data, and who has to be reachable.
flowchart LR
subgraph CS["Client-server"]
C1[Client A] --> S[(Server)]
C2[Client B] --> S
C3[Client C] --> S
end
subgraph P2P["Peer-to-peer"]
P1[Peer 1] <--> P2[Peer 2]
P2 <--> P3[Peer 3]
P3 <--> P1
P4[Peer 4] <--> P2
end
| Property | Client-server | Peer-to-peer |
|---|---|---|
| Who initiates | Clients always connect to the server | Any peer may connect to any other |
| Reachability | Server needs a stable, public address | Every peer ideally reachable, which NAT makes hard |
| Scaling | Server capacity is the ceiling; you buy more servers | Capacity grows with the number of peers |
| Control | Central: auth, logging, policy in one place | Distributed: hard to moderate, hard to shut down |
| Failure | Server down means service down | Individual peers churn constantly; the swarm survives |
| Examples | HTTP, SMTP, IMAP, SSH, FTP, DNS lookups | BitTorrent, WebRTC media, blockchain gossip, IPFS |
Pure models are rare. BitTorrent uses a central tracker (or a DHT) to find peers, then transfers data peer-to-peer. WebRTC uses a signalling server, then sends media directly between browsers; see WebRTC. Email is client-server at each hop but server-to-server between domains, which makes the MTAs peers of one another.
The single biggest practical difference is who must accept inbound connections. A server listens on a known port. A peer behind a home router usually cannot accept inbound connections at all, which is why P2P systems spend so much effort on NAT traversal; see NAT.
Sockets and Ports: a Recap
A process does not talk to "a computer." It talks to a socket, an endpoint the operating system identifies by an address and a port. A TCP connection is uniquely identified by the 5-tuple:
(protocol, source IP, source port, destination IP, destination port)
(TCP, 192.0.2.10, 51544, 203.0.113.5, 443)
The server binds a well-known port (0 to 1023, needing root or CAP_NET_BIND_SERVICE on Linux), clients get an ephemeral port chosen by the OS (on Linux, net.ipv4.ip_local_port_range, default 32768 to 60999). Ports 1024 to 49151 are "registered" (PostgreSQL 5432, Redis 6379), and IANA's ephemeral range is 49152 to 65535.
# Which processes are listening, and on which ports?
sudo ss -tulpn
# Which ports does this machine think belong to which service?
grep -E '^(smtp|imap|pop3|ftp|ssh|ntp|snmp|ldap)\s' /etc/services
# Linux ephemeral port range
sysctl net.ipv4.ip_local_port_range
The socket API is the same for every protocol on this page:
import socket
# A TCP client: the whole of "speaking a protocol" is send() and recv()
with socket.create_connection(("example.com", 80), timeout=5) as s:
s.sendall(b"HEAD / HTTP/1.1\r\nHost: example.com\r\nConnection: close\r\n\r\n")
print(s.recv(4096).decode())
A text protocol (HTTP/1.1, SMTP, POP3, IMAP, FTP) sends human-readable commands terminated by CRLF (\r\n), which is why you can debug it with telnet, nc or openssl s_client. A binary protocol (SSH, HTTP/2, DNS, SNMP, NTP, LDAP) needs a dedicated client or Wireshark.
The three debugging clients you need
nc host port: a raw TCP pipe, no TLS.nc -ufor UDP.openssl s_client -connect host:port -crlf: a TLS pipe. Add-starttls smtp|imap|pop3|ftp|ldapfor protocols that upgrade a plaintext connection.curl -v: speaks HTTP, FTP, SMTP, IMAP, POP3, LDAP and more, and prints the conversation.
Common Design Choices Across Protocols
Once you have seen three text protocols you have seen them all. They make the same few choices, and comparing them is a classic exam question.
| Design choice | Options | Examples |
|---|---|---|
| State | Stateless (each request standalone) vs stateful (session state on server) | HTTP is stateless; FTP and IMAP are stateful |
| Control channel | In-band (commands and data on one connection) vs out-of-band (separate connections) | HTTP, SMTP in-band; FTP out-of-band |
| Push vs pull | Sender pushes to receiver vs receiver pulls from store | SMTP pushes; POP3, IMAP and HTTP pull |
| Connection lifetime | Per-request vs persistent | HTTP/1.0 per-request; HTTP/1.1, IMAP persistent |
| Encoding | Text (CRLF lines) vs binary (length-prefixed frames) | SMTP text; SSH, HTTP/2 binary |
| Transport | TCP (reliability) vs UDP (latency, simplicity) | SMTP, SSH on TCP; DNS, NTP, SNMP, TFTP, DHCP on UDP |
| Security | Implicit TLS (dedicated port) vs STARTTLS (upgrade) | IMAPS 993 implicit; SMTP 587 STARTTLS |
Part 2: HTTP/1.1 at a Glance
HTTP is the protocol you know best, so this is deliberately brief. The binary framing, multiplexing and QUIC transport of later versions live in HTTP/2 and HTTP/3; the encryption underneath lives in The TLS Handshake.
A Raw Exchange
$ printf 'GET /index.html HTTP/1.1\r\nHost: example.com\r\nUser-Agent: nc\r\nConnection: close\r\n\r\n' \
| nc example.com 80
HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Content-Length: 1256
Cache-Control: max-age=604800
Date: Sun, 08 Nov 2026 10:12:03 GMT
Connection: close
<!doctype html>
...
A request is a request line (method, target, version), headers, a blank line, and an optional body. A response is a status line, headers, a blank line, and a body. The Host header is mandatory in HTTP/1.1; it is what lets one IP address serve thousands of virtual hosts.
Methods
| Method | Purpose | Safe | Idempotent | Body |
|---|---|---|---|---|
| GET | Retrieve a representation | Yes | Yes | No (semantically undefined) |
| HEAD | GET without the body | Yes | Yes | No |
| POST | Process data, create subordinate resource | No | No | Yes |
| PUT | Replace the resource at this URI | No | Yes | Yes |
| PATCH | Partially modify | No | Not guaranteed | Yes |
| DELETE | Remove | No | Yes | Optional |
| OPTIONS | Ask what is allowed (CORS preflight) | Yes | Yes | Optional |
Safe means no server-side state change is intended. Idempotent means doing it twice has the same effect as once, which is what makes retries safe.
Status Codes
| Class | Meaning | Ones you must know |
|---|---|---|
| 1xx | Informational | 100 Continue, 101 Switching Protocols (WebSocket upgrade) |
| 2xx | Success | 200 OK, 201 Created, 204 No Content, 206 Partial Content |
| 3xx | Redirection | 301 Moved Permanently, 302 Found, 304 Not Modified, 307/308 (method-preserving) |
| 4xx | Client error | 400, 401 Unauthorized (really "unauthenticated"), 403 Forbidden, 404, 409 Conflict, 429 Too Many Requests |
| 5xx | Server error | 500, 502 Bad Gateway, 503 Service Unavailable, 504 Gateway Timeout |
Headers Worth Knowing
| Header | Direction | Purpose |
|---|---|---|
Host | Request | Which virtual host |
Content-Type, Content-Length | Both | Body format and size |
Transfer-Encoding: chunked | Response | Body sent in chunks when length is unknown up front |
Accept, Accept-Encoding | Request | Content negotiation (gzip, br) |
Cache-Control, ETag, If-None-Match | Both | Caching and conditional requests (304) |
Authorization | Request | Credentials (Bearer ..., Basic ...) |
Set-Cookie / Cookie | Response / Request | State on top of a stateless protocol |
Connection | Both | keep-alive or close |
Persistent Connections
HTTP/1.0 opened a new TCP connection per request: a three-way handshake (and, with TLS, a TLS handshake) for every image. HTTP/1.1 made connections persistent by default: the connection stays open until one side sends Connection: close or an idle timeout fires.
Pipelining (sending several requests without waiting) was also specified, but responses must come back in order, so one slow response blocks everything behind it: application-layer head-of-line blocking. Browsers disabled pipelining and instead open about six parallel connections per origin. That workaround is exactly the problem HTTP/2 multiplexing solves.
Non-persistent (HTTP/1.0): per object = 2 RTT (TCP handshake + request/response)
Persistent (HTTP/1.1): first object = 2 RTT, each later object = 1 RTT
Cookies: State on a Stateless Protocol
HTTP keeps no memory between requests. Cookies bolt state on: the server sets a value, and the browser returns it on every later request to that site.
HTTP/1.1 200 OK
Set-Cookie: sid=4f9a1c; Path=/; HttpOnly; Secure; SameSite=Lax; Max-Age=3600
GET /account HTTP/1.1
Host: shop.example
Cookie: sid=4f9a1c
HttpOnly keeps JavaScript from reading it, Secure sends it only over HTTPS, and SameSite limits cross-site sending (the main CSRF defence). Sessions built on cookies are covered in Session Management.
Part 3: Email
Email is the oldest widely used application on the internet and the best showcase of application-layer design: a push protocol (SMTP) to move mail between servers, and separate pull protocols (POP3, IMAP) for users to read it. It is also a store-and-forward system: every hop takes full responsibility for the message before the previous hop deletes it.
The Cast of Agents
| Agent | Full name | Job | Examples |
|---|---|---|---|
| MUA | Mail User Agent | The client you read and write in | Thunderbird, Outlook, Apple Mail, Gmail web UI |
| MSA | Mail Submission Agent | Accepts mail from authenticated users (port 587 or 465) | Postfix submission service, Exchange |
| MTA | Mail Transfer Agent | Relays mail between servers (port 25) | Postfix, Exim, Sendmail, Microsoft Exchange |
| MDA | Mail Delivery Agent | Writes the message into the recipient's mailbox | Dovecot LDA/LMTP, procmail |
| MAA / mail store | Mail Access Agent | Serves the mailbox over IMAP or POP3 | Dovecot, Cyrus, Exchange |
How a Message Travels
flowchart LR
A["Alice's MUA<br/>(Thunderbird)"] -->|"SMTP submission<br/>port 587 + AUTH"| MSA["MSA<br/>mail.alice.example"]
MSA --> MTA1["Outbound MTA<br/>(queue)"]
MTA1 -->|"DNS MX lookup<br/>for bob.example"| DNS[("DNS")]
MTA1 -->|"SMTP relay<br/>port 25"| MTA2["Inbound MTA<br/>mx1.bob.example"]
MTA2 -->|"SPF / DKIM / DMARC<br/>spam filtering"| MDA["MDA<br/>(LMTP to store)"]
MDA --> BOX[("Bob's mailbox")]
BOX -->|"IMAP 993<br/>or POP3 995"| B["Bob's MUA"]
Step by step:
- Alice's MUA connects to her provider's MSA on port 587, upgrades to TLS, authenticates, and submits the message with SMTP.
- The MSA hands it to the outbound MTA, which queues it. The message is now the server's responsibility; Alice's laptop can go offline.
- The MTA looks up the MX records for
bob.examplein DNS and picks the lowest preference value. - It opens an SMTP connection to Bob's inbound MTA on port 25 (server-to-server; no user authentication, opportunistic STARTTLS).
- The inbound MTA checks SPF, DKIM and DMARC, runs spam filters, and passes the message to the MDA, usually over LMTP.
- The MDA writes it to Bob's mailbox (Maildir or a database).
- Bob's MUA fetches it over IMAP (or POP3).
If step 4 fails with a temporary error (4xx), the sending MTA keeps the message queued and retries with backoff, typically for up to five days before bouncing. That is why email is reliable despite unreliable servers.
MX Lookup
MX records map a domain to the hosts that accept its mail, each with a preference (lower is tried first).
$ dig +short MX gmail.com
5 gmail-smtp-in.l.google.com.
10 alt1.gmail-smtp-in.l.google.com.
20 alt2.gmail-smtp-in.l.google.com.
30 alt3.gmail-smtp-in.l.google.com.
40 alt4.gmail-smtp-in.l.google.com.
$ dig +short A gmail-smtp-in.l.google.com
142.250.153.27
If a domain has no MX record, RFC 5321 says to fall back to its A/AAAA record (the "implicit MX"). A domain that sends no mail and wants none should publish a null MX (MX 0 ., RFC 7505). How DNS resolution works end to end is in DNS Deep Dive.
SMTP: a Full Session
SMTP (RFC 5321) is a text protocol: the client sends four-letter commands, the server replies with a three-digit code and text. Here is a real-shaped submission session to an MSA, using openssl s_client to handle STARTTLS (lines from the client are prefixed C: and server lines S: for readability; the tool does not print those).
openssl s_client -starttls smtp -connect smtp.alice.example:587 -crlf -quiet
S: 220 smtp.alice.example ESMTP Postfix
C: EHLO laptop.alice.example
S: 250-smtp.alice.example
S: 250-PIPELINING
S: 250-SIZE 52428800
S: 250-AUTH PLAIN LOGIN
S: 250-ENHANCEDSTATUSCODES
S: 250-8BITMIME
S: 250 SMTPUTF8
C: AUTH PLAIN AGFsaWNlQGFsaWNlLmV4YW1wbGUAaHVudGVyMg==
S: 235 2.7.0 Authentication successful
C: MAIL FROM:<alice@alice.example>
S: 250 2.1.0 Ok
C: RCPT TO:<bob@bob.example>
S: 250 2.1.5 Ok
C: RCPT TO:<carol@bob.example>
S: 250 2.1.5 Ok
C: DATA
S: 354 End data with <CR><LF>.<CR><LF>
C: From: Alice <alice@alice.example>
C: To: Bob <bob@bob.example>
C: Subject: Lunch?
C: Date: Sun, 08 Nov 2026 12:00:00 +0530
C: Message-ID: <20261108120000.1@alice.example>
C:
C: Thursday at noon?
C: .
S: 250 2.0.0 Ok: queued as 4QwXyZ1kLmz
C: QUIT
S: 221 2.0.0 Bye
What each step means:
| Command | Meaning |
|---|---|
EHLO | "Extended hello": identify yourself and ask for the server's extensions. Plain HELO is the pre-1995 version with no extensions |
STARTTLS | Upgrade this plaintext connection to TLS (done for you by -starttls smtp); you must EHLO again afterwards |
AUTH PLAIN | Base64 of \0username\0password. Only acceptable inside TLS |
MAIL FROM | The envelope sender, also called return-path: where bounces go |
RCPT TO | One envelope recipient per command; repeat for each |
DATA | Everything until a line containing only . is the message (headers + body) |
QUIT | Close politely |
The base64 credential above was produced like this (do not do this with a real password in your shell history):
printf '\0alice@alice.example\0hunter2' | base64
A message body line that starts with . is "dot-stuffed" by the client (sent as ..) so it cannot be mistaken for the terminator; the server strips the extra dot.
SMTP reply codes follow a pattern worth memorising:
| First digit | Meaning | Sender's response |
|---|---|---|
| 2xx | Success | Continue |
| 3xx | Intermediate, send more (354 after DATA) | Continue |
| 4xx | Transient failure (421 service unavailable, 450 mailbox busy, 451 greylisted) | Keep in queue, retry later |
| 5xx | Permanent failure (550 no such user, 552 too big, 554 rejected) | Bounce to the envelope sender |
The 4xx vs 5xx distinction is what makes greylisting work: a server deliberately answers 451 to an unknown sender, real MTAs retry a few minutes later and get through, and many spam bots never retry.
Envelope vs Headers
The single most important idea in email: the envelope and the headers are independent.
flowchart TB
subgraph ENV["SMTP envelope (used for routing)"]
E1["MAIL FROM alice@alice.example"]
E2["RCPT TO bob@bob.example<br/>RCPT TO carol@bob.example"]
end
subgraph MSG["Message (shown to the user)"]
H["Headers<br/>From, To, Cc, Subject, Date"]
BODY["Body<br/>(MIME parts)"]
end
ENV --> MSG
The envelope is what the MTAs act on. The headers are just text inside DATA that the MUA displays. They can disagree:
- Bcc works because the Bcc recipient appears only in an envelope
RCPT TO, never in the headers. - Mailing lists keep
From: alice@...in the header but change the envelope sender to the list's bounce address. - Spoofing works because nothing in basic SMTP stops
MAIL FROM:<x@y>andFrom: ceo@yourbank.examplefrom being anything at all.
Each MTA that handles a message prepends a Received: header, so reading them bottom to top traces the path. The receiving server records the envelope sender as Return-Path:.
POP3 vs IMAP
SMTP only pushes. Reading mail is a separate problem with two classic answers.
POP3 (Post Office Protocol v3, RFC 1939) is a "download and (usually) delete" protocol. Port 110, or 995 for implicit TLS.
openssl s_client -connect pop.alice.example:995 -crlf -quiet
S: +OK Dovecot ready.
C: USER alice
S: +OK
C: PASS hunter2
S: +OK Logged in.
C: STAT
S: +OK 2 4823
C: LIST
S: +OK 2 messages:
S: 1 1967
S: 2 2856
S: .
C: RETR 1
S: +OK 1967 octets
S: Return-Path: <bob@bob.example>
S: ... full message ...
S: .
C: DELE 1
S: +OK Marked to be deleted.
C: QUIT
S: +OK Logging out, messages deleted.
Replies are only +OK or -ERR. Deletion is marked during the session and applied at QUIT (the UPDATE state); drop the connection without QUIT and nothing is deleted.
IMAP (Internet Message Access Protocol, RFC 9051 for IMAP4rev2) keeps mail on the server and lets clients manipulate it remotely: folders, flags (\Seen, \Flagged, \Deleted), server-side search, partial fetch of a single attachment, and idle push. Port 143, or 993 for implicit TLS. Every command carries a client-chosen tag so responses can be matched to commands.
openssl s_client -connect imap.alice.example:993 -crlf -quiet
S: * OK [CAPABILITY IMAP4rev1 IDLE AUTH=PLAIN] Dovecot ready.
C: a1 LOGIN alice hunter2
S: a1 OK Logged in
C: a2 LIST "" "*"
S: * LIST (\HasNoChildren) "/" INBOX
S: * LIST (\HasNoChildren \Sent) "/" Sent
S: * LIST (\HasNoChildren \Trash) "/" Trash
S: a2 OK List completed
C: a3 SELECT INBOX
S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
S: * 12 EXISTS
S: * OK [UIDVALIDITY 1699420000] UIDs valid
S: a3 OK [READ-WRITE] Select completed
C: a4 SEARCH UNSEEN FROM "bob"
S: * SEARCH 11 12
S: a4 OK Search completed
C: a5 FETCH 12 (FLAGS BODY.PEEK[HEADER.FIELDS (FROM SUBJECT DATE)])
S: * 12 FETCH (FLAGS () BODY[HEADER.FIELDS (FROM SUBJECT DATE)] {78}
S: From: Bob <bob@bob.example>
S: Subject: Re: Lunch?
S: Date: Sun, 08 Nov 2026 12:40:00 +0000
S: )
S: a5 OK Fetch completed
C: a6 STORE 12 +FLAGS (\Seen)
S: * 12 FETCH (FLAGS (\Seen))
S: a6 OK Store completed
C: a7 LOGOUT
S: * BYE Logging out
S: a7 OK Logout completed
Lines starting with * are untagged responses (data); the tagged line ends the command. BODY.PEEK fetches without setting \Seen. {78} is a literal: the next 78 bytes are raw data.
| Property | POP3 | IMAP |
|---|---|---|
| Model | Download, then usually delete | Mail lives on the server |
| Multiple devices | Poor: whichever device downloads first wins | Designed for it: same state everywhere |
| Folders | Only the inbox | Full folder hierarchy |
| Flags (read, starred) | None on server | Synced on server |
| Partial fetch | TOP n lines only | Any MIME part, any header subset |
| Server search | No | Yes (SEARCH) |
| Push | No, client polls | IDLE holds the connection for new-mail notifications |
| Server storage | Small | Large; quotas matter |
| State on server | Almost none | Significant (per-folder UIDs, flags) |
| Ports | 110, 995 (TLS) | 143, 993 (TLS) |
Modern webmail (Gmail, Outlook.com) uses proprietary HTTP APIs internally, and JMAP (RFC 8620) is an HTTP/JSON successor to IMAP, but IMAP remains the universal interoperability protocol.
MIME: Making SMTP Carry Anything
Original SMTP carried only 7-bit ASCII text with lines under 1000 characters. MIME (Multipurpose Internet Mail Extensions, RFC 2045 to 2049) adds headers that describe what the body actually is, so the same plain-text transport carries HTML, attachments and non-English text.
From: Alice <alice@alice.example>
To: Bob <bob@bob.example>
Subject: =?UTF-8?B?UmVwb3J0IOKAkyBRMw==?=
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="b1_XyZ"
--b1_XyZ
Content-Type: multipart/alternative; boundary="b2_AbC"
--b2_AbC
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Hi Bob, the Q3 report is attached. Caf=C3=A9 on Thursday?
--b2_AbC
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<p>Hi Bob, the <b>Q3 report</b> is attached. Caf=C3=A9 on Thursday?</p>
--b2_AbC--
--b1_XyZ
Content-Type: application/pdf; name="q3.pdf"
Content-Disposition: attachment; filename="q3.pdf"
Content-Transfer-Encoding: base64
JVBERi0xLjcKJcfsj6IKNSAwIG9iago8PC9MZW5ndGggNiAwIFIvRmlsdGVyIC9GbGF0ZURlY29kZT4+
...
--b1_XyZ--
| MIME header | Purpose |
|---|---|
MIME-Version: 1.0 | Declares the message as MIME |
Content-Type | Media type: text/plain, text/html, image/png, application/pdf, multipart/* |
boundary | The separator string between parts of a multipart body |
Content-Transfer-Encoding | How the bytes were made 7-bit safe: 7bit, quoted-printable, base64, 8bit, binary |
Content-Disposition | inline or attachment, plus a filename |
Encoded words =?UTF-8?B?...?= | Non-ASCII text in headers (B = base64, Q = quoted-printable) |
The multipart types to know: multipart/mixed (body plus attachments), multipart/alternative (same content in plain and HTML; client shows the best one it can render, last is preferred), and multipart/related (HTML plus inline images referenced by cid:).
Base64 turns every 3 bytes into 4 ASCII characters, so attachments grow by about 33%, plus line breaks. That is why a 10 MB attachment limit really means about 7.5 MB of file. Quoted-printable leaves ASCII readable and escapes other bytes as =XX, which is efficient for mostly-English text.
MIME types escaped from email and became the web's Content-Type header; HTTP reuses the same registry.
Email Authentication at a Glance
Because the envelope and the From: header can be forged, three DNS-published mechanisms were layered on top. They are covered in depth, with record syntax and rollout plans, in Email Security & Deliverability; here is the one-screen summary.
| Mechanism | Question it answers | Published as | Checks |
|---|---|---|---|
| SPF | Is this IP allowed to send for the envelope domain? | TXT v=spf1 include:_spf.google.com -all | Connecting IP vs MAIL FROM domain |
| DKIM | Was this message signed by the domain, and unaltered? | TXT at selector._domainkey.domain | Signature in DKIM-Signature: header |
| DMARC | Does SPF or DKIM pass and align with the visible From:? What to do if not? | TXT at _dmarc.domain | Alignment + policy none/quarantine/reject + reports |
dig +short TXT gmail.com | grep spf
dig +short TXT _dmarc.gmail.com
Transport encryption between MTAs is opportunistic STARTTLS by default, which an active attacker can strip. MTA-STS and DANE make it mandatory; again see the email security page.
Email Ports
| Port | Protocol | Use |
|---|---|---|
| 25 | SMTP | Server-to-server relay. Blocked outbound by most ISPs and clouds to stop spam bots |
| 587 | SMTP submission | Client to MSA, STARTTLS + AUTH (RFC 6409) |
| 465 | SMTPS | Client to MSA, implicit TLS (re-legitimised by RFC 8314, now preferred) |
| 110 / 995 | POP3 / POP3S | Mail retrieval |
| 143 / 993 | IMAP / IMAPS | Mail access |
Port 25 from the cloud
AWS, GCP and Azure block or throttle outbound port 25 by default. If your app "cannot send email" from a new VM, that is usually why. Send through a relay (SES, SendGrid, Postmark) on 587 or 465, or through their HTTP APIs.
Part 4: File Transfer
FTP: Two Connections
FTP (RFC 959, 1985) predates the modern internet's NAT and firewalls, and its design shows it. It uses two separate TCP connections:
- The control connection to server port 21 carries commands (
USER,PASS,LIST,RETR,STOR) and replies, and stays open for the whole session. This is out-of-band control. - A fresh data connection is opened for each file transfer or directory listing, then closed.
FTP is stateful: the server remembers the current directory, the logged-in user and the transfer mode for the session. HTTP, by contrast, sends everything in each request.
The interesting question is who opens the data connection.
Active Mode
In active mode the server connects back to the client.
sequenceDiagram
participant C as Client 192.0.2.10
participant S as FTP server
C->>S: TCP connect to port 21 (control)
S-->>C: 220 Service ready
C->>S: USER alice / PASS secret
S-->>C: 230 Logged in
C->>S: PORT 192,0,2,10,195,80 (listen on 50000)
S-->>C: 200 PORT command successful
C->>S: RETR report.csv
S->>C: TCP connect FROM port 20 TO client port 50000
S-->>C: 150 Opening data connection
S->>C: file bytes on data connection
S-->>C: 226 Transfer complete
The PORT argument encodes the IP and port as six numbers: h1,h2,h3,h4,p1,p2, where port = p1 × 256 + p2. Here 195 × 256 + 80 = 50000.
Passive Mode
In passive mode the client connects to the server for data too.
sequenceDiagram
participant C as Client behind NAT
participant S as FTP server 203.0.113.5
C->>S: TCP connect to port 21 (control)
S-->>C: 220 Service ready
C->>S: USER alice / PASS secret
S-->>C: 230 Logged in
C->>S: PASV
S-->>C: 227 Entering Passive Mode (203,0,113,5,234,98)
C->>S: TCP connect to port 60002 (data)
C->>S: RETR report.csv
S->>C: file bytes on data connection
S-->>C: 226 Transfer complete
234 × 256 + 98 = 60002. The modern EPSV (RFC 2428) returns only a port, 229 Entering Extended Passive Mode (|||60002|), and works over IPv6.
Why NAT and Firewalls Break Active Mode
Active mode has three separate problems:
- Inbound connection to the client. A client behind NAT or a stateful firewall does not accept unsolicited inbound connections. The server's connection from port 20 is dropped.
- Private address in the payload. The client sends
PORT 192,168,1,23,..., its private address, inside the application data. NAT rewrites IP headers, not payloads, so the server tries to connect to an unroutable address. - Unpredictable ports. Firewalls cannot know in advance which high port to open, so rules based on ports alone do not work.
Workarounds exist: NAT devices and firewalls with an FTP application-layer gateway (ALG) read the control channel, rewrite PORT commands and open "related" pinholes (Linux nf_conntrack_ftp). But the ALG cannot read an encrypted control channel, which is why FTPS through NAT is especially painful.
Passive mode fixes the client side, since all connections are outbound from the client, and moves the problem to the server: it must open a range of passive ports in its firewall and, if itself behind NAT, advertise its public IP in the 227 reply.
# vsftpd.conf: passive mode behind a cloud NAT
pasv_enable=YES
pasv_min_port=60000
pasv_max_port=60100
pasv_address=203.0.113.5
| Active | Passive | |
|---|---|---|
| Who opens the data connection | Server (from port 20) | Client |
| Client firewall/NAT | Must allow inbound: usually breaks | Outbound only: works |
| Server firewall | Only 21 inbound, 20 outbound | 21 plus a passive port range inbound |
| Default in modern clients | No | Yes (browsers, curl, FileZilla) |
Other FTP details that come up in exams: ASCII vs binary transfer types (TYPE A converts line endings and corrupts binary files; TYPE I is image/binary), and the fact that FTP sends the password in cleartext on the control connection.
# curl speaks FTP; -v shows the control-channel conversation
curl -v --ftp-pasv -u alice:secret ftp://ftp.example.com/pub/report.csv -o report.csv
SFTP vs FTPS vs SCP
Because plain FTP sends credentials in cleartext, three secure alternatives exist. Their names are confusingly similar and they are entirely different protocols.
| FTPS | SFTP | SCP | |
|---|---|---|---|
| What it is | FTP wrapped in TLS | A file-transfer subsystem of SSH | A copy command over SSH |
| Related to FTP? | Yes, same commands | No, a different binary protocol | No |
| Ports | 21 explicit (AUTH TLS) or 990 implicit, plus data ports | 22 only | 22 only |
| Connections | Control + data (same NAT problems as FTP, worse) | One | One |
| Authentication | Password, or X.509 client certs | SSH keys or password | SSH keys or password |
| Operations | Full FTP: list, rename, delete, resume | Full: list, rename, delete, resume, permissions | Copy only |
| Firewall friendliness | Poor | Excellent | Excellent |
| Typical use | Legacy B2B, banks, compliance mandates | The default for secure file exchange | Quick one-off copies |
# SFTP: interactive or batch
sftp -i ~/.ssh/deploy_ed25519 deploy@files.example.com
sftp> put build.tar.gz /incoming/
sftp> ls -l /incoming
# SCP: OpenSSH 9.0+ actually uses the SFTP protocol underneath by default; -O forces legacy SCP
scp -i ~/.ssh/deploy_ed25519 build.tar.gz deploy@files.example.com:/incoming/
# rsync over SSH: delta transfer, the usual choice for syncing directories
rsync -avz -e "ssh -i ~/.ssh/deploy_ed25519" ./dist/ deploy@files.example.com:/var/www/app/
Use SFTP or rsync, not SCP, for anything new
The legacy SCP protocol runs the remote shell with the path as an argument, which led to path-injection bugs (CVE-2019-6111 and others). OpenSSH 9.0 switched the scp command to SFTP underneath. Prefer sftp or rsync in scripts.
TFTP
Trivial FTP (RFC 1350) is FTP stripped to the bone: UDP port 69, no authentication, no directory listings, just read (RRQ) and write (WRQ) requests. Data moves in 512-byte blocks, each acknowledged before the next is sent (stop-and-wait). A block shorter than 512 bytes marks the end of the file.
Client -> server:69 RRQ "pxelinux.0" mode "octet"
Server -> client DATA block 1 (512 bytes)
Client -> server ACK block 1
Server -> client DATA block 2 (512 bytes)
...
Server -> client DATA block N (< 512 bytes: last)
Client -> server ACK block N
The server replies from a new ephemeral port for the transfer, which is another NAT/firewall headache. It exists because it is tiny enough to live in a network card's boot ROM: PXE network boot fetches its bootloader over TFTP, and switches and IP phones pull firmware and configs from it. Never expose TFTP to the internet; anyone can read (and maybe write) files.
Part 5: Remote Access
Telnet: the Cautionary Tale
Telnet (RFC 854, port 23) gives you a remote terminal over a plain TCP connection. Everything, including the username and password, crosses the network in cleartext. Anyone on the path with tcpdump sees your password:
sudo tcpdump -i eth0 -A 'tcp port 23'
# ... l.o.g.i.n.:. .r.o.o.t ... P.a.s.s.w.o.r.d.:. .t.o.o.r ...
Telnet as a remote shell is obsolete, but the telnet client survives as a debugging tool ("can I open a TCP connection to port 5432?"), and Telnet servers still ship on old network gear and IoT devices, which is how the Mirai botnet recruited hundreds of thousands of cameras in 2016 using default passwords. Today nc -vz host port does the reachability check better.
SSH
SSH (Secure Shell, RFC 4251 to 4254, port 22) replaced Telnet, rlogin and rsh in the late 1990s. It provides an encrypted, integrity-protected, server-authenticated channel, and then multiplexes many things over it: shells, commands, file transfer (SFTP), and forwarded TCP connections.
The Three Layers
flowchart TB
subgraph CONN["Connection protocol (RFC 4254)"]
CH1["Channel: session<br/>(shell, exec, sftp subsystem)"]
CH2["Channel: direct-tcpip<br/>(local forward)"]
CH3["Channel: forwarded-tcpip<br/>(remote forward)"]
end
AUTH["User authentication protocol (RFC 4252)<br/>publickey, password, keyboard-interactive"]
TRANS["Transport layer protocol (RFC 4253)<br/>key exchange, server host key, encryption, MAC"]
TCP["TCP port 22"]
CONN --> AUTH --> TRANS --> TCP
| Layer | Job |
|---|---|
| Transport | Negotiate algorithms, run key exchange, authenticate the server, then encrypt and integrity-protect every packet. Supports periodic re-keying |
| User authentication | Authenticate the user to the server, inside the encrypted transport |
| Connection | Multiplex many independent channels over the one connection, each with its own flow-control window |
The Handshake and Key Exchange
sequenceDiagram
participant C as Client
participant S as Server
C->>S: TCP connect port 22
S-->>C: SSH-2.0-OpenSSH_9.6
C->>S: SSH-2.0-OpenSSH_9.8
C->>S: KEXINIT (supported kex, host key, cipher, MAC algorithms)
S-->>C: KEXINIT (server lists)
Note over C,S: Both pick the first client choice the server supports
C->>S: KEX_ECDH_INIT (client ephemeral public key)
S-->>C: KEX_ECDH_REPLY (host public key, server ephemeral key, signature over exchange hash)
Note over C: Verify host key against known_hosts and check the signature
C->>S: NEWKEYS
S-->>C: NEWKEYS
Note over C,S: Everything from here is encrypted with keys derived from the shared secret
C->>S: SERVICE_REQUEST ssh-userauth
C->>S: USERAUTH_REQUEST publickey (signature with user key)
S-->>C: USERAUTH_SUCCESS
C->>S: CHANNEL_OPEN session, then exec or shell
- Version exchange in plaintext (
SSH-2.0-...). SSH-1 is broken and long gone. - Algorithm negotiation (
KEXINIT): key exchange (for examplecurve25519-sha256or the post-quantum hybridmlkem768x25519-sha256, default in OpenSSH 10), host key type (ssh-ed25519,rsa-sha2-256), cipher (chacha20-poly1305@openssh.com,aes256-gcm@openssh.com), MAC. - Ephemeral Diffie-Hellman (elliptic curve): each side sends a fresh public value; both compute the same shared secret. Because the keys are ephemeral, recording the traffic and stealing the server's key later does not decrypt it: forward secrecy.
- The server signs the exchange hash with its long-term host key. This is what proves you are talking to the real server and not a man in the middle.
- Session keys for each direction are derived from the shared secret and the exchange hash.
It is the same idea as the TLS 1.3 handshake, with one big difference: TLS authenticates servers with certificates signed by CAs, while SSH by default uses trust on first use of a raw public key. The cryptographic background is in Symmetric vs Asymmetric Encryption and The TLS Handshake.
# Watch the negotiation
ssh -vv user@host 2>&1 | grep -E 'kex: algorithm|host key algorithm|cipher|Authenticat'
# Which algorithms does my client support?
ssh -Q kex; ssh -Q cipher; ssh -Q key
Host Keys and known_hosts
The first time you connect, the client has no way to know whether the host key is genuine:
The authenticity of host 'build01.example.com (203.0.113.20)' can't be established.
ED25519 key fingerprint is SHA256:Rj8B1yX3...Qe4.
Are you sure you want to continue connecting (yes/no/[fingerprint])?
Answering yes stores the key in ~/.ssh/known_hosts. On every later connection, a mismatch produces the famous warning:
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
That is either a man-in-the-middle attack or, far more often, a rebuilt server or a reused IP. Verify out of band before running ssh-keygen -R host.
Better than trust-on-first-use at scale:
# On the server: print its host key fingerprint to compare out of band
ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub
# Pre-seed known_hosts from a trusted network position (still TOFU, but once)
ssh-keyscan -t ed25519 build01.example.com >> ~/.ssh/known_hosts
# Publish SSHFP records in DNSSEC-signed DNS and let the client check them
ssh-keygen -r build01.example.com # prints SSHFP records
ssh -o VerifyHostKeyDNS=yes build01.example.com
The most scalable answer is SSH certificates: an internal CA signs host keys (and user keys), clients trust the CA with one @cert-authority line in known_hosts, and no per-host prompts ever appear.
User Authentication
| Method | How it works | Notes |
|---|---|---|
publickey | Client proves possession of a private key by signing session data; server checks ~/.ssh/authorized_keys | The default for automation. Ed25519 keys preferred |
password | Password sent inside the encrypted channel | Vulnerable to guessing; disable on internet-facing hosts |
keyboard-interactive | Server-driven prompts | Used for OTP/MFA via PAM |
| Certificates | User key signed by a CA with principals and expiry | Short-lived certs remove authorized_keys sprawl |
FIDO2 (ed25519-sk) | Private key in a hardware token, touch required | Phishing-resistant |
ssh-keygen -t ed25519 -C "alice@laptop" -f ~/.ssh/id_ed25519
ssh-copy-id -i ~/.ssh/id_ed25519.pub alice@build01.example.com
A minimal hardening baseline for /etc/ssh/sshd_config:
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
AllowGroups ssh-users
MaxAuthTries 3
AllowTcpForwarding no # re-enable per Match block where tunnels are needed
X11Forwarding no
Run sshd -t to validate the config before reloading so you do not lock yourself out.
Port Forwarding and Tunnels (Ops View)
The connection layer can carry arbitrary TCP streams. This is how operators reach private databases and internal dashboards without exposing them, and also how attackers pivot, which is why it should be allowed deliberately rather than by default.
flowchart LR
L["Laptop<br/>localhost:5433"] -->|"encrypted SSH<br/>port 22"| B["Bastion host"]
B -->|"plain TCP<br/>inside VPC"| DB[("Postgres<br/>10.0.3.7:5432")]
| Type | Command | Meaning | Ops use |
|---|---|---|---|
Local (-L) | ssh -L 5433:10.0.3.7:5432 bastion | Connections to my localhost:5433 come out of the bastion to 10.0.3.7:5432 | Reach a private database or admin UI |
Remote (-R) | ssh -R 8080:localhost:3000 public-host | Connections to public-host:8080 come back to my localhost:3000 | Expose a local dev server; also a classic exfiltration path |
Dynamic (-D) | ssh -D 1080 bastion | A SOCKS5 proxy; each connection goes wherever the client asks | Browse internal sites via the bastion |
Jump (-J) | ssh -J bastion app01 | Hop through a bastion to a host behind it | Replaces agent forwarding for multi-hop access |
# Reach a private RDS instance through a bastion, no shell (-N), in background (-f)
ssh -f -N -L 5433:mydb.abc123.us-east-1.rds.amazonaws.com:5432 ec2-user@bastion.example.com
psql "host=localhost port=5433 user=app dbname=app"
# ~/.ssh/config: make the jump permanent
Host bastion
HostName bastion.example.com
User ec2-user
IdentityFile ~/.ssh/id_ed25519
Host app-*
ProxyJump bastion
User deploy
Agent forwarding is a loaded gun
ssh -A lets the remote host use your local SSH agent. Anyone with root on that host can then authenticate as you to anything your keys open, for as long as you are connected. Prefer ProxyJump, which never exposes the agent to the intermediate host.
On the defensive side, restrict forwarding per user or key (permitopen="10.0.3.7:5432" in authorized_keys, PermitOpen and AllowTcpForwarding local in a Match block), log forwarding events, and treat long-lived reverse tunnels to unknown hosts as an incident signal. Many teams now replace bastions entirely with identity-aware proxies (AWS SSM Session Manager, Teleport, Tailscale SSH); see Network Segmentation.
| Telnet | SSH | |
|---|---|---|
| Port | 23 | 22 |
| Encryption | None | Yes (AES-GCM, ChaCha20-Poly1305) |
| Server authentication | None | Host key, verified via known_hosts or CA |
| User authentication | Cleartext password | Public key, certificates, password, MFA |
| Integrity | None | MAC/AEAD on every packet |
| Multiplexing | One terminal | Many channels: shells, SFTP, forwards |
| Today | Debugging only, legacy devices | Universal |
Part 6: The Supporting Cast
DNS (Recap)
DNS translates names to addresses and, as you saw with MX, much more. It mostly runs over UDP port 53 for queries (falling back to TCP for large responses and zone transfers), with DoT (853) and DoH (443) for encrypted lookups. The hierarchy, caching, TTLs and record types are in DNS Deep Dive.
dig +trace www.example.com # walk root -> TLD -> authoritative
DHCP (Pointer)
DHCP hands a booting machine its IP address, subnet mask, gateway and DNS servers through the four-message DORA exchange (Discover, Offer, Request, Acknowledge) over UDP 67 (server) and 68 (client). It is an application-layer protocol by packaging, but it exists to configure the network layer, so it is covered alongside IP addressing in The Network Layer.
SNMP
Simple Network Management Protocol lets a monitoring system (the manager) read and change values on devices (the agents): interface byte counters on a switch, temperature on a UPS, toner level on a printer. Values live in a tree called the MIB (Management Information Base), each identified by an OID such as 1.3.6.1.2.1.1.3.0 (sysUpTime).
| Operation | Direction | Port |
|---|---|---|
Get, GetNext, GetBulk | Manager polls agent | UDP 161 |
Set | Manager changes agent value | UDP 161 |
Trap / Inform | Agent pushes an alert to the manager unprompted | UDP 162 |
# Walk the system subtree of a switch (SNMPv2c, community "public")
snmpwalk -v2c -c public 10.0.0.1 1.3.6.1.2.1.1
# SNMPv2-MIB::sysDescr.0 = STRING: Cisco IOS Software, C2960 ...
# DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (412093422) 47 days, 16:42:14.22
# SNMPv3 with authentication and privacy
snmpget -v3 -l authPriv -u monitor -a SHA -A 'authpass' -x AES -X 'privpass' 10.0.0.1 sysUpTime.0
| Version | Security |
|---|---|
| v1, v2c | A cleartext "community string" acting as a shared password; public/private defaults are notorious |
| v3 | Real user authentication (HMAC) and encryption (AES); use this |
SNMP is why it uses UDP: a monitoring poll should not tie up connection state on a struggling device, and a lost poll is simply retried next interval. Modern stacks increasingly use streaming telemetry (gNMI over gRPC) and Prometheus exporters (snmp_exporter bridges the two).
NTP and Why Clocks Matter
Network Time Protocol (RFC 5905, UDP 123) keeps clocks synchronised to within milliseconds over the internet and well under a millisecond on a LAN. Servers are arranged in strata: stratum 0 is a reference clock (GPS, atomic), stratum 1 servers are attached to one, stratum 2 sync from stratum 1, and so on.
The core calculation uses four timestamps from one request-response:
t1 = client send time t2 = server receive time
t3 = server send time t4 = client receive time
offset θ = ((t2 - t1) + (t3 - t4)) / 2
delay δ = (t4 - t1) - (t3 - t2)
This assumes the network delay is symmetric; any asymmetry becomes offset error of up to half the asymmetry. Example: t1 = 100.000, t2 = 100.120, t3 = 100.121, t4 = 100.041 (seconds). Then θ = (0.120 + 0.080) / 2 = 0.100 s (the client is 100 ms behind) and δ = 0.041 − 0.001 = 0.040 s.
chronyc tracking # current offset, stratum, reference
chronyc sources -v # which servers, reachability, jitter
timedatectl # is NTP sync active?
Why it matters for software:
| Failure | What goes wrong |
|---|---|
| Clock ahead or behind by minutes | TLS certificates appear not-yet-valid or expired; JWT exp/nbf checks fail; TOTP codes rejected |
| Kerberos skew over 5 minutes | Active Directory logins fail outright |
| Skew between nodes | Log correlation lies about event order; last-write-wins replication drops the newer write |
| Clock jumps backwards | Timeouts and rate limiters computed from wall time misfire; use monotonic clocks for durations |
| Lease or lock expiry across machines | Two nodes both believe they hold a lease; see fencing tokens |
These consequences, plus PTP, TrueTime and logical clocks, are covered in Clock Synchronization and Vector Clocks & Lamport Timestamps.
LDAP
Lightweight Directory Access Protocol (RFC 4511, TCP 389, or 636 for LDAPS) queries and modifies a directory: a hierarchical database of users, groups and devices, optimised for reads. Each entry has a distinguished name (DN) such as uid=alice,ou=people,dc=example,dc=com and attributes defined by schema object classes.
# Find Alice's groups (StartTLS with -ZZ)
ldapsearch -H ldap://ldap.example.com -ZZ \
-D "cn=readonly,dc=example,dc=com" -W \
-b "ou=people,dc=example,dc=com" "(uid=alice)" cn mail memberOf
| Operation | Purpose |
|---|---|
| Bind | Authenticate (simple bind = DN + password, or SASL/Kerberos) |
| Search | Base DN + scope + filter like (&(objectClass=person)(mail=*@example.com)) |
| Add, Modify, Delete, ModifyDN | Change entries |
Microsoft Active Directory is an LDAP directory plus Kerberos; see Active Directory. Applications typically use LDAP to authenticate employees, though modern SSO puts SAML or OIDC in front of it; see Enterprise SSO. Simple bind on port 389 without StartTLS sends the password in cleartext, and unescaped user input in search filters is LDAP injection.
Peer-to-Peer and BitTorrent Basics
BitTorrent distributes a large file by splitting it into pieces (typically 256 KB to 4 MB) and having every downloader also upload the pieces it already has.
flowchart TB
T["Tracker or DHT<br/>(who has this torrent?)"]
S["Seeder<br/>(has all pieces)"]
P1["Peer A<br/>pieces 1, 4"]
P2["Peer B<br/>pieces 2, 3"]
P3["Peer C<br/>(new)"]
P3 -->|"announce info-hash"| T
T -->|"peer list"| P3
S <--> P1
S <--> P2
P1 <-->|"exchange pieces"| P2
P3 <--> P1
P3 <--> P2
| Concept | Meaning |
|---|---|
.torrent file / magnet link | Metadata: file names, piece size, and the SHA-1 (v1) or SHA-256 (v2) hash of each piece; identified by the info-hash |
| Tracker | A server that returns a list of peers for an info-hash |
| DHT | A Kademlia distributed hash table so peers can find each other without a tracker |
| Swarm | All peers sharing a torrent; seeders have everything, leechers are still downloading |
| Rarest first | Request pieces fewest peers have, so no piece disappears from the swarm |
| Tit-for-tat (choking) | Upload preferentially to peers that upload to you; one "optimistic unchoke" slot tries new peers |
The scaling argument is the classic exam numerical. To distribute a file of size F to N peers, with server upload rate u_s, peer upload rates u_i and slowest peer download rate d_min:
Client-server: D_cs >= max( N·F / u_s , F / d_min )
P2P: D_p2p >= max( F / u_s , F / d_min , N·F / (u_s + Σ u_i) )
Client-server time grows linearly with N. P2P time is bounded as N grows, because every new peer brings upload capacity as well as demand.
Real-Time and RPC Protocols (Pointers)
| Protocol | One line | Page |
|---|---|---|
| WebSockets | Full-duplex messages over a single TCP connection, upgraded from HTTP via 101 Switching Protocols | WebSockets |
| gRPC | RPC with Protocol Buffers over HTTP/2 streams | gRPC Internals |
| MQTT | Lightweight publish/subscribe over TCP for IoT, port 1883 / 8883 | MQTT |
| WebRTC | Peer-to-peer media and data in browsers over UDP | WebRTC |
| QUIC | Encrypted, multiplexed transport over UDP that carries HTTP/3 | QUIC Protocol |
Reference Tables
Well-Known Ports
| Port | Transport | Protocol | Notes |
|---|---|---|---|
| 20, 21 | TCP | FTP data, control | Data port 20 only in active mode |
| 22 | TCP | SSH, SFTP, SCP | |
| 23 | TCP | Telnet | Cleartext; avoid |
| 25 | TCP | SMTP relay | Server to server |
| 53 | UDP, TCP | DNS | TCP for large responses and zone transfers |
| 67, 68 | UDP | DHCP server, client | |
| 69 | UDP | TFTP | |
| 80 | TCP | HTTP | |
| 110 | TCP | POP3 | |
| 123 | UDP | NTP | |
| 143 | TCP | IMAP | |
| 161, 162 | UDP | SNMP, SNMP traps | |
| 389 | TCP | LDAP | |
| 443 | TCP, UDP | HTTPS, HTTP/3 (QUIC) | |
| 465 | TCP | SMTP submission, implicit TLS | |
| 514 | UDP | Syslog | |
| 587 | TCP | SMTP submission, STARTTLS | |
| 636 | TCP | LDAPS | |
| 853 | TCP | DNS over TLS | |
| 989, 990 | TCP | FTPS data, control (implicit) | |
| 993 | TCP | IMAPS | |
| 995 | TCP | POP3S | |
| 1883, 8883 | TCP | MQTT, MQTT over TLS | |
| 3306 | TCP | MySQL | Registered, not well-known |
| 3389 | TCP | RDP | Windows remote desktop |
| 5432 | TCP | PostgreSQL | Registered |
| 6379 | TCP | Redis | Registered |
| 6881 to 6889 | TCP, UDP | BitTorrent (traditional) | Clients now pick random ports |
Protocol Comparison
| Protocol | Transport | Port(s) | State | Push or pull | Format | Secure variant |
|---|---|---|---|---|---|---|
| HTTP/1.1 | TCP | 80 | Stateless | Pull | Text | HTTPS (443) |
| SMTP | TCP | 25, 587, 465 | Stateful session | Push | Text | STARTTLS, SMTPS |
| POP3 | TCP | 110 | Stateful session | Pull | Text | POP3S (995) |
| IMAP | TCP | 143 | Stateful, server-side mail state | Pull (+ IDLE push) | Text | IMAPS (993) |
| FTP | TCP | 21 + data | Stateful, out-of-band control | Both | Text control | FTPS, or use SFTP |
| SFTP | TCP | 22 | Stateful | Both | Binary | Always encrypted |
| TFTP | UDP | 69 | Minimal | Both | Binary | None |
| Telnet | TCP | 23 | Stateful | Interactive | Text | Use SSH |
| SSH | TCP | 22 | Stateful | Interactive | Binary | Always encrypted |
| DNS | UDP/TCP | 53 | Stateless | Pull | Binary | DoT, DoH |
| DHCP | UDP | 67/68 | Lease state | Broadcast pull | Binary | None (snooping on switches) |
| SNMP | UDP | 161/162 | Stateless | Pull (+ trap push) | Binary (ASN.1 BER) | SNMPv3 |
| NTP | UDP | 123 | Stateless | Pull | Binary | NTS (RFC 8915) |
| LDAP | TCP | 389 | Stateful (bind) | Pull | Binary (ASN.1 BER) | LDAPS, StartTLS |
| BitTorrent | TCP/UDP | Varies | Per-peer | Both | Binary | Protocol encryption (weak) |
Common Misconceptions
| Misconception | Reality |
|---|---|
| SFTP is FTP over SSL | SFTP is an SSH subsystem with its own binary protocol. FTP over TLS is FTPS |
The From: header decides where bounces go | Bounces go to the envelope MAIL FROM (Return-Path) |
| IMAP is just a newer POP3 | They have different models: IMAP manages mail on the server, POP3 downloads it |
| SSH is secure, so the first connection is too | The first connection trusts whatever key is presented unless you verify the fingerprint, use SSHFP or a CA |
| Passive FTP has no firewall issues | It moves the problem to the server, which must open a passive port range |
| HTTP/1.1 pipelining gives parallelism | Responses still arrive in order; one slow response blocks the rest |
| NTP is only cosmetic | Skewed clocks break TLS, Kerberos, JWT expiry, TOTP and last-write-wins replication |
| Port 465 is deprecated | It was briefly, but RFC 8314 (2018) recommends implicit TLS on 465 for submission |
Practice Problems
Problem 1. A web page has a base HTML file and 10 small images on the same server. The RTT is 50 ms and transmission time is negligible. How long does the full page take with (a) non-persistent HTTP and serial connections, (b) persistent HTTP without pipelining, (c) persistent HTTP with pipelining?
Solution
Each new TCP connection costs 1 RTT for the handshake, and each request-response costs 1 RTT.
(a) Non-persistent, serial: 11 objects × 2 RTT = 22 RTT = 1100 ms.
(b) Persistent, no pipelining: the first object costs 2 RTT (handshake + request). Each of the 10 images then costs 1 RTT on the same connection: 2 + 10 = 12 RTT = 600 ms.
(c) Persistent with pipelining: 2 RTT for the HTML, then all 10 image requests are sent back to back and their responses return in about 1 RTT: 2 + 1 = 3 RTT = 150 ms.
(TLS would add 1 RTT per new connection with TLS 1.3, 2 with TLS 1.2.)
Problem 2. An FTP client sends PORT 10,0,0,5,4,1. Which address and port will the server connect to, and why will this probably fail on the internet?
Solution
The address is 10.0.0.5 and the port is 4 × 256 + 1 = 1025. This is active mode, so the server opens the connection to the client. 10.0.0.5 is a private RFC 1918 address, unroutable on the internet, and the client is almost certainly behind NAT that will not accept unsolicited inbound connections anyway. The fix is passive mode (PASV/EPSV), where the client opens the data connection.
Problem 3. A server replies 227 Entering Passive Mode (198,51,100,7,195,149). What does the client do next?
Solution
It opens a new TCP connection to 198.51.100.7 on port 195 × 256 + 149 = 49920 + 149 = 50069, then sends the transfer command (for example RETR) on the control connection. The data flows over the 50069 connection.
Problem 4. You want to send a 9 MB PDF by email. The receiving server's limit is 10 MB (SIZE 10485760). Will it go through?
Solution
Base64 expands data by 4/3: 9 MB × 4/3 = 12 MB, plus about 2.7% for CRLF line breaks every 76 characters, so roughly 12.3 MB. That exceeds 10 MB, so the server will reject it (typically 552 5.3.4 Message size exceeds fixed limit). The largest file that fits is roughly 10 / 1.37 ≈ 7.3 MB. Share a link instead.
Problem 5. A mail log shows 451 4.7.1 Greylisted, please try again later. Should the application alert someone? What happens to the message?
Solution
No alert is needed. 4xx is a transient failure; the sending MTA keeps the message queued and retries, usually within minutes, at which point the greylisting server recognises the (sender IP, envelope sender, recipient) triple and accepts it. Only alert if the message is still queued after many hours, or on 5xx permanent failures, which generate bounces.
Problem 6. An NTP client records t1 = 10.000, t2 = 10.540, t3 = 10.541, t4 = 10.061 (seconds). Compute offset and round-trip delay. What is the maximum error if the path were completely asymmetric?
Solution
θ = ((t2 − t1) + (t3 − t4)) / 2 = (0.540 + 0.480) / 2 = 0.510 s, so the client is about 510 ms slow.
δ = (t4 − t1) − (t3 − t2) = 0.061 − 0.001 = 0.060 s.
The true offset lies within θ ± δ/2, so the error bound is ±30 ms. Lower delay means a tighter bound, which is why NTP prefers the samples with the smallest delay.
Problem 7. A 1 GB file must reach N = 100 peers. The server upload rate is 100 Mbps, every peer uploads at 10 Mbps and downloads at 100 Mbps. Compare the minimum distribution times for client-server and P2P.
Solution
F = 1 GB = 8000 Mb.
Client-server: max(N·F / u_s, F / d_min) = max(100 × 8000 / 100, 8000 / 100) = max(8000, 80) = 8000 s (about 2.2 hours).
P2P: max(F / u_s, F / d_min, N·F / (u_s + N·u)) = max(80, 80, 800000 / (100 + 1000)) = max(80, 80, 727) = ≈ 727 s (about 12 minutes).
At N = 1000, client-server takes 80000 s while P2P takes 8000000 / 10100 ≈ 792 s: almost unchanged.
Problem 8. Users report that ssh build01 now prints "REMOTE HOST IDENTIFICATION HAS CHANGED". What do you do?
Solution
Do not blindly run ssh-keygen -R. First find out whether the change is expected: was the host rebuilt, reimaged, or moved to a new IP behind the same name? Get the new fingerprint out of band (the cloud console's serial output, configuration management, or ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub via a trusted session) and compare it with the one shown. If it matches, remove the old entry and accept the new key. If it does not, treat it as a possible man-in-the-middle and escalate. Long term, preserve host keys across rebuilds or adopt SSH host certificates so rebuilds never trigger the warning.
Problem 9. Write the ssh command that lets you run psql on your laptop against a database at 10.20.0.15:5432 that is reachable only from bastion.corp.example, without opening a shell. Which sshd setting could block it?
Solution
ssh -N -L 5432:10.20.0.15:5432 alice@bastion.corp.example
psql -h localhost -p 5432 -U app appdb
If something already listens locally on 5432, pick another local port such as 5433. The bastion's AllowTcpForwarding no (or PermitOpen not including 10.20.0.15:5432, or a permitopen= restriction on Alice's key in authorized_keys) would block it.
Problem 10. A user reads mail on a phone and a laptop. Messages read on the phone still show as unread on the laptop, and some messages are missing entirely from the laptop. Diagnose.
Solution
The phone (or both) is configured for POP3 with "delete from server after download". The device that polls first downloads and deletes the message, so the other never sees it, and POP3 has no server-side \Seen flag to sync read state. Switch both devices to IMAP (993), which keeps mail and flags on the server.
Problem 11. Classify each as TCP or UDP and explain why: DNS query, SMTP, TFTP, SNMP poll, SSH, NTP.
Solution
- DNS query: UDP (one small request, one small reply; retries are cheap). TCP when the reply is large or for zone transfers.
- SMTP: TCP (multi-step stateful conversation, must not lose message data).
- TFTP: UDP (simple enough to fit in boot ROM; it does its own stop-and-wait reliability).
- SNMP poll: UDP (no connection state on stressed devices; a lost poll is retried next interval).
- SSH: TCP (interactive, ordered byte stream, every byte matters).
- NTP: UDP (a single timestamp exchange; TCP retransmission delays would corrupt the timing measurement).
Problem 12. An email arrives with From: ceo@company.example, but its headers show Return-Path: <bounce@mailer.evil.example>, spf=pass smtp.mailfrom=mailer.evil.example, dkim=none, dmarc=fail. Explain each result.
Solution
SPF checked the envelope domain (mailer.evil.example), whose own SPF record authorises the sending IP, so SPF passes. But SPF's domain does not align with the visible From: domain (company.example), and there is no DKIM signature from company.example, so DMARC, which requires an aligned pass, fails. If company.example publishes p=reject the receiving server should reject the message. This is exactly the gap DMARC was designed to close; see Email Security.
Interview Questions
1. Walk me through what happens when you send an email from Gmail to an Outlook address.
The MUA submits via HTTPS (webmail) or SMTP on 587/465 to Google's MSA after authentication. Google's MTA queues the message, signs it with DKIM, looks up the MX records for the recipient domain, connects to the lowest-preference MX on port 25, negotiates STARTTLS (enforced if MTA-STS is published), and transfers the message with MAIL FROM, RCPT TO, DATA. Microsoft's inbound MTA checks SPF, DKIM and DMARC, runs filtering, and delivers to the mailbox via its MDA. The recipient's client then fetches it over IMAP, Exchange ActiveSync or a web API. Temporary failures (4xx) are retried from the queue; permanent failures (5xx) bounce to the envelope sender.
2. What is the difference between the SMTP envelope and the message headers?
The envelope (MAIL FROM, RCPT TO) is the routing instruction the MTAs use; the headers (From:, To:, Cc:) are part of the message content shown to the user. They are independent. Bcc works by having a recipient only in the envelope; bounces go to the envelope sender; spoofing exploits the fact that nothing in base SMTP ties From: to anything. SPF validates the envelope domain, and DMARC requires alignment with the header domain.
3. POP3 or IMAP: which and why? IMAP for almost everyone: mail and its flags live on the server, so multiple devices stay in sync, and folders, server-side search and partial fetch work. POP3 suits a single device that wants a local archive and minimal server storage. POP3 is also simpler to implement.
4. Explain active vs passive FTP and why passive is the default.
FTP uses a control connection on 21 and separate data connections. In active mode the client sends PORT and the server connects back to the client; in passive mode the client sends PASV, the server replies with an IP and port, and the client connects. Active mode fails behind NAT and client firewalls because it needs an inbound connection to the client and embeds a private address in the payload. Passive keeps all connections outbound from the client, at the cost of the server opening a passive port range.
5. Compare SFTP, FTPS and SCP. FTPS is FTP over TLS and inherits FTP's two-connection design and firewall pain. SFTP is an SSH subsystem: one connection on port 22, key-based authentication, full file operations. SCP is a copy-only command over SSH, whose legacy protocol had injection bugs; modern OpenSSH runs it over SFTP. Choose SFTP (or rsync over SSH) unless a partner mandates FTPS.
6. How does SSH protect against a man-in-the-middle, and where is it weak?
The server signs the key-exchange hash with its host key, and the client checks that key against known_hosts. An attacker cannot forge the signature without the server's private key. The weakness is the first connection: by default the client trusts whatever key it is shown (trust on first use). Mitigations are verifying fingerprints out of band, SSHFP records in DNSSEC-signed zones, or SSH host certificates signed by an internal CA.
7. What are SSH local, remote and dynamic forwarding?
Local (-L) makes a port on your machine reach a destination via the SSH server, used to reach private databases. Remote (-R) makes a port on the SSH server reach back to a destination via your machine, used to expose a local service. Dynamic (-D) runs a SOCKS proxy through the server. Operationally, forwarding should be restricted (AllowTcpForwarding, PermitOpen), and reverse tunnels are a common exfiltration and persistence technique, so log and alert on them.
8. Why do most protocols on this page use TCP, while DNS, NTP, SNMP, DHCP and TFTP use UDP? TCP gives ordered, reliable streams, which conversations with many steps and large payloads need. UDP suits single small request-response exchanges where a retry is cheaper than connection setup (DNS, SNMP), where timing accuracy matters more than delivery (NTP), where the client has no IP address yet and must broadcast (DHCP), or where the implementation must be tiny (TFTP in boot ROM).
9. Why does clock synchronisation matter in a distributed system? Certificates, tokens and one-time codes are validated against wall time; Kerberos rejects skew over five minutes; logs from different hosts cannot be ordered; last-write-wins replication silently drops the newer write if the older writer's clock runs fast; and leases can overlap. NTP keeps skew in the millisecond range, but systems that need correctness should use fencing tokens, logical clocks or bounded-uncertainty clocks rather than trust wall time alone.
10. Why is HTTP called stateless when every website has logins? The protocol itself carries no memory between requests; each request must contain everything the server needs. Applications add state on top with cookies, tokens in headers, or URL parameters, and the server looks up the session. That statelessness is what lets any server behind a load balancer handle any request.
11. What does MIME solve, and what does it cost?
SMTP was designed for 7-bit ASCII text. MIME adds Content-Type, Content-Transfer-Encoding and multipart bodies so email can carry HTML, any character set and binary attachments. The cost is encoding overhead: base64 inflates binary data by about 33%, and the nested structure is complex enough that MIME parsing bugs remain a recurring source of security issues.
12. How does BitTorrent scale better than a single server? Every downloader also uploads the pieces it has, so total upload capacity grows with demand. Client-server distribution time grows linearly with the number of clients; P2P distribution time stays roughly constant. Rarest-first piece selection keeps all pieces available, and tit-for-tat choking discourages free riding.
Summary
| Topic | Key idea |
|---|---|
| Architectures | Client-server centralises control; P2P scales capacity with participants but fights NAT |
| Sockets | A connection is a 5-tuple; servers bind well-known ports, clients use ephemeral ports |
| HTTP/1.1 | Stateless text request/response; persistent connections; cookies add state |
| SMTP | Push, store-and-forward, envelope separate from headers, 4xx retry vs 5xx bounce |
| POP3 vs IMAP | Download-and-delete vs server-side mailbox with folders and flags |
| MIME | Content types, multipart bodies, base64 at about 33% overhead |
| FTP | Separate control and data connections; active breaks behind NAT, passive is the default |
| SFTP / FTPS / SCP | SSH subsystem / FTP over TLS / copy over SSH; prefer SFTP or rsync |
| SSH | Transport, auth, connection layers; ephemeral ECDH, host keys, forwarding |
| SNMP, NTP, LDAP | Device monitoring over UDP; clock sync that everything secretly depends on; the directory behind corporate logins |
Where to Go Next
- OSI Model & TCP/IP Stack: where these protocols sit
- TCP/IP Deep Dive and UDP: the transports underneath
- HTTP/2 and HTTP/3: what replaced HTTP/1.1's limitations
- The TLS Handshake: how SMTPS, IMAPS, FTPS and HTTPS are encrypted
- DNS Deep Dive: MX, TXT and the resolver chain
- Email Security & Deliverability: SPF, DKIM, DMARC, MTA-STS in depth
- NAT: why active FTP and P2P are hard
- Clock Synchronization: NTP internals, PTP and TrueTime
- Network Debugging: tcpdump, Wireshark and friends
- Networking Fundamentals for Security: the same ports through an attacker's eyes