TCP in Depth: Header, Flow Control and Congestion Control

By the end of this lesson you will be able to

• Read every field of a TCP header and of a UDP header, and say what each one is for and what a strange value in it means.
• Explain how TCP detects and repairs loss: sequence and acknowledgment numbers, RTT estimation and the retransmission timeout, duplicate ACKs, fast retransmit and fast recovery, SACK.
• Explain flow control with the receive window, zero windows, window probes and window scaling.
• Explain congestion control: cwnd, slow start (correctly), ssthresh, congestion avoidance, AIMD, and how Reno, CUBIC and BBR differ; say what ECN adds.
• Calculate a bandwidth-delay product and predict when one connection cannot fill a link.
• Recognise the Nagle and delayed-ACK stall, and know when to use TCP_NODELAY, TCP Fast Open and keepalive.
• Turn all of this into system-design decisions: connection reuse, buffer tuning, BBR on servers, and why QUIC moved transport into user space.

#Where this lesson fits

Lesson 1.9 showed TCP from the outside: the handshake, sequence numbers, a lost segment, closing, TLS and QUIC. This lesson opens the box. Everything here is still the transport layer (layer 4), whose job is process-to-process (end-to-end) delivery: getting data from one program on one machine to the right program on another.

The transport layer does eight jobs. TCP does all of them; UDP does only the first three in its simplest form:

The unit of data has a different name per protocol: a TCP segment, a UDP datagram. TCP’s promise is a full-duplex, reliable, ordered byte stream: both sides send at once, nothing is lost, nothing is reordered, and there are no message boundaries (two writes of 100 bytes may arrive as one read of 200).

#TCP vs UDP, one more time

#The TCP header, field by field

Every TCP segment starts with a header of 20 bytes plus 0–40 bytes of options, so 20–60 bytes. It is drawn in rows of 32 bits:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-------------------------------+-------------------------------+
|          Source Port          |       Destination Port        |
+-------------------------------+-------------------------------+
|                        Sequence Number                        |
+---------------------------------------------------------------+
|                    Acknowledgment Number                      |
+-------+-------+-+-+-+-+-+-+-+-+-------------------------------+
| Data  |Reser- |C|E|U|A|P|R|S|F|                               |
|Offset | ved   |W|C|R|C|S|S|Y|I|            Window             |
|       |       |R|E|G|K|H|T|N|N|                               |
+-------+-------+-+-+-+-+-+-+-+-+-------------------------------+
|           Checksum            |        Urgent Pointer         |
+-------------------------------+-------------------------------+
|                Options (0–40 bytes) + padding                 |
+---------------------------------------------------------------+
|                         Data (payload)                        |
+---------------------------------------------------------------+

#The eight flags

ECE and CWR were added in 2001 for Explicit Congestion Notification (ECN), covered below. An older ninth flag, NS, was declared historic and is now one of the reserved bits.

#Options worth knowing

• MSS (maximum segment size), SYN only: the largest payload this side can receive, 1460 bytes for IPv4 on Ethernet (1500 − 20 − 20) and 1440 for IPv6. Not the same as the MTU: MSS counts only TCP data.
• Window scale, SYN only: a shift count 0–14. Every later window value is multiplied by 2^shift, so windows can reach about 1 GB instead of 64 KB.
• SACK-permitted (SYN) and SACK blocks (later): the receiver can list which byte ranges beyond a hole it already holds.
• Timestamps (10 bytes, 12 with padding): each side stamps its clock and echoes the other’s. This gives an RTT sample on every ACK and protects against wrapped sequence numbers (PAWS): a 32-bit sequence number wraps after 4 GB, which at 10 Gbps takes only about 3.4 seconds.
• NOP and end of list: one-byte fillers that keep options aligned.
• Fast Open cookie: lets a repeat client put data in the SYN (see below).

Options agreed in the SYN can never be added later. If the SYN-ACK does not echo window scaling or SACK, that feature is off for the whole connection.

#The UDP header

UDP needs only 8 bytes, always:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-------------------------------+-------------------------------+
|          Source Port          |       Destination Port        |
+-------------------------------+-------------------------------+
|            Length             |           Checksum            |
+-------------------------------+-------------------------------+

What UDP leaves out is the whole point: no sequence numbers, no acknowledgments, no window, no connection state. Each datagram is an independent message. Anything more (ordering, retries, rate control) is the application’s job, which is exactly what DNS (query ID and retry) and QUIC (its own packet numbers and ACK frames, encrypted inside the UDP payload) do.

#Opening and closing, briefly

Lesson 1.9 walks through the handshake with real numbers and failures; here is the short version in the header’s own terms.

 Client                                           Server (LISTEN)
   | ---- SYN        seq=x ------------------------> |
   | <--- SYN, ACK   seq=y, ack=x+1 ---------------- |
   | ---- ACK        ack=y+1 ----------------------> |
 [ESTABLISHED]                                 [ESTABLISHED]

1. SYN (flag SYN = 1): the client says “I want to talk to you; my starting sequence number is x.”
2. SYN-ACK (SYN = 1 and ACK = 1): the server says “I got your x, send x+1 next (ACK); and my starting sequence number is y (SYN).”
3. ACK (ACK = 1): the client confirms “I got your y, I expect y+1 next. Connected!”

The options (MSS, window scale, SACK-permitted, timestamps) ride on the two SYNs. Closing is a four-way exchange, one FIN and one ACK per direction: FIN → ACK → FIN → ACK. Because each direction closes separately, one side can stop sending while still receiving (a half-close), and the second FIN often rides in the same segment as the ACK, so a capture frequently shows three segments, not four. The side that closed first then waits in TIME_WAIT (60 seconds on Linux) so that late duplicates of old segments die out. All of this is animated in lesson 1.9.

#Reliability in depth

#Sequence and acknowledgment numbers

TCP numbers bytes, not packets. A segment with seq=1001 and 1448 bytes of data carries bytes 1001–2448; the receiver answers ack=2449, meaning “I have everything up to 2448”. Three consequences matter:

• ACKs are cumulative. One ACK can confirm many segments, and a lost ACK is harmless if a later one arrives.
• A receiver can only acknowledge up to the first hole. Data after a hole is kept in the buffer but not acknowledged by the ack number. That is what makes duplicate ACKs possible.
• The numbers wrap. After 2³² bytes (4 GB) the counter starts again from zero. Timestamps (PAWS) stop an old delayed segment from being mistaken for new data after a wrap.

#The retransmission timeout and RTT estimation

When TCP sends data it starts a retransmission timer. If no ACK covers that data before the retransmission timeout (RTO), it resends. Too short a timeout resends data that was only slow; too long a timeout leaves the connection idle after a real loss. So TCP measures the round trip continuously:

• SRTT (smoothed RTT): SRTT = 7/8 × SRTT + 1/8 × new sample.
• RTTVAR (how much the RTT varies): RTTVAR = 3/4 × RTTVAR + 1/4 × |SRTT − sample|.
• RTO = SRTT + 4 × RTTVAR, with a floor (the standard says 1 s, Linux uses 200 ms) and a ceiling (120 s on Linux). Before any sample, the RTO is 1 second.

Karn’s rule: never take an RTT sample from a segment that was retransmitted, because you cannot tell which copy the ACK is for. Timestamps solve this, since the echoed timestamp names the copy. After each timeout the RTO doubles (exponential backoff). Linux abandons an established connection after 15 failed retransmissions, roughly 15 minutes, which is why applications need their own, shorter timeouts.

#Duplicate ACKs, fast retransmit and fast recovery

Waiting for the timer is slow, so TCP has a quicker signal. Say packets 1, 3 and 4 arrive but 2 is lost. The receiver answers each later arrival with the same ACK, “I still want packet 2”: these are duplicate ACKs, and they are sent immediately. After three duplicate ACKs the sender concludes that packet 2 is lost but that the network is still delivering:

• Fast retransmit: resend the missing segment at once, before the timer fires.
• Fast recovery: do not fall back to one segment. Set ssthresh to half the data in flight and continue from there in congestion avoidance (Reno also inflates cwnd by three segments during recovery to account for the segments that left the network, then deflates it). Each extra duplicate ACK means another segment has left the network, so new data can keep flowing.

Why three and not one? Packets are sometimes reordered on the way; one or two duplicates are normal and do not prove loss. Three is the classic compromise. Modern Linux adds RACK, which judges loss by time (“a segment sent later has already been acknowledged, and enough time has passed”), and a tail loss probe for the case where the last segments of a burst are lost and there is nothing after them to produce duplicate ACKs.

#SACK: say exactly what is missing

A cumulative ACK can only describe the first hole. With two losses in one window, plain TCP (NewReno) repairs one hole per round trip. With SACK, each duplicate ACK also carries up to four blocks (three with timestamps) of data received beyond the hole, for example ack 2001, SACK 3001–4000, 5001–7000. The sender keeps a scoreboard and resends exactly the missing pieces in one round trip. SACK is on by default almost everywhere.

#Flow control: the receive window

Flow control matches the sender’s speed to the receiver’s capacity. If a fast sender pushes data faster than the receiving application reads it, the receiver’s buffer overflows and data is dropped. TCP prevents that with a sliding window.

• Every ACK carries the receive window (rwnd) in the header’s window field: the free space in the receiver’s socket buffer right now.
• The sender may have at most rwnd bytes sent but not yet acknowledged.
• As the application reads, space frees up; as ACKs arrive, the window’s left edge moves forward. The window slides along the byte stream.

  bytes:  ... |  sent and ACKed  |  sent, not ACKed  |  may send now  |  not yet allowed ...
                                 ^ left edge (last ack)        right edge = ack + rwnd ^

The sender is actually limited by two windows: it may have in flight at most min(rwnd, cwnd). rwnd protects the receiver; cwnd (next section) protects the network.

#Zero window and window probes

If the application stops reading (a long garbage-collection pause, a stuck database call, a slow disk), the buffer fills and the receiver advertises window = 0 (a zero window). The sender stops sending data and starts a persist timer. When it fires, the sender sends a tiny zero window probe to make the receiver answer with its current window, and backs off the interval each time the answer is still zero. Without probes, a lost window update (a pure ACK, which TCP never retransmits) would leave both sides waiting forever.

A related trap is silly window syndrome: a receiver that advertises every few freed bytes invites tiny segments that are mostly header. Receivers fix it by not reopening the window until at least one MSS or half the buffer is free; senders fix it with Nagle’s algorithm.

#Window scaling and auto-tuning

The window field has only 16 bits, so without help the window tops out at 65,535 bytes. That was plenty in 1981 and is far too small today. The window scale option, agreed in the SYNs, multiplies the field by up to 2¹⁴, allowing windows of about 1 GB. Operating systems then auto-tune the buffer, growing it while the application keeps up (Linux’s default maximum receive buffer is about 6 MB, set by net.ipv4.tcp_rmem).

#Congestion control: the congestion window

Flow control works between the sender and the receiver. Congestion control protects everything in between: the network path and its routers. When more packets arrive at a router than its outgoing link can carry, its queue fills and it starts dropping packets. TCP detects this and slows down.

This is not optional politeness. In October 1986 the early internet suffered a congestion collapse: links were full of retransmissions of retransmissions, and throughput between two sites in California fell by a factor of almost a thousand. Van Jacobson’s algorithms of 1988 (slow start, congestion avoidance, fast retransmit) fixed it, and every TCP since then has a congestion window (cwnd): the sender’s own estimate of how much data the path can hold.

The classic (Reno) algorithm has four parts.

#Slow start

A new connection does not know the path’s capacity, so it probes:

• It starts with an initial window of 10 segments, about 14 KB (RFC 6928; Linux since 2011). Old stacks used 1–4 segments.
• For every ACK that acknowledges new data, cwnd grows by one segment. A whole window of ACKs comes back each round trip, so cwnd doubles every RTT: 10 → 20 → 40 → 80. Despite the name, growth is exponential.
• Slow start stops when cwnd reaches the slow start threshold (ssthresh), or when a loss happens. ssthresh starts “infinitely” high, so the very first slow start ends with a loss.

#Congestion avoidance

Once cwnd ≥ ssthresh, TCP stops doubling and grows by only one segment per round trip (linear, or additive increase). It is carefully probing for spare capacity.

#Loss detection and recovery

TCP treats a lost packet as a sign of congestion and reacts in two ways, depending on how the loss is detected:

1. Timeout (RTO expired), serious congestion. No ACKs at all came back. Set ssthresh = half the data in flight, set cwnd = 1 segment, and start again with slow start.
2. Three duplicate ACKs, mild loss. Some segments are still arriving. Fast retransmit the missing segment, then fast recovery: ssthresh = half, cwnd = ssthresh, and continue in congestion avoidance instead of starting from one.

Growing by one segment per RTT and halving on loss is called AIMD (additive increase, multiplicative decrease). Its graph is the famous sawtooth: up slowly, down by half, up slowly again. AIMD has a useful property: several flows sharing a link converge to roughly equal shares, which is how millions of unrelated connections share the internet fairly without any central coordinator.

#Reno vs CUBIC vs BBR

Reno is the textbook model. Real systems run its successors:

The key insight: loss is not always congestion. On Wi-Fi or a long intercontinental link a packet may be lost to noise. A loss-based algorithm halves its rate anyway; BBR, which models the path instead, keeps going. That is why large content providers run BBR on their servers.

#ECN: a warning instead of a drop

With Explicit Congestion Notification, a router whose queue is building up marks a packet (“congestion experienced”, two bits in the IP header) instead of dropping it. The receiver echoes the mark back with the ECE flag, the sender reduces cwnd exactly as for a loss and confirms with CWR. Nothing has to be retransmitted and nobody waits for a timer, so latency stays low. ECN is negotiated in the handshake and must be supported by both ends and by the routers; data centres use it heavily (DCTCP), and the internet is adopting it gradually, most recently for L4S (low-latency, low-loss, scalable throughput).

#Bandwidth-delay product and long fat pipes

How much data must be “in the air” to keep a link busy? Think of the path as a pipe: its width is the bandwidth, its length is the round-trip time. The volume is the bandwidth-delay product (BDP):

BDP = bandwidth × RTT

A worked example, a 1 Gbps link from Asia to the US with a 200 ms round trip:

1. 1 Gbps × 0.2 s = 200,000,000 bits = 25,000,000 bytes = 25 MB.
2. To fill the link, TCP must keep 25 MB unacknowledged in flight, so both the receive window and the congestion window must reach at least 25 MB.
3. With a 64 KB window (no window scaling): one window per round trip = 65,535 bytes / 0.2 s ≈ 328 KB/s ≈ 2.6 Mbps. That is 0.26% of the link. The sender spends 99.7% of each round trip idle, waiting for ACKs.
4. Throughput of one connection is at most window / RTT, so to reach 1 Gbps at 200 ms you need a 25 MB window; at 1 ms (inside a data centre) 125 KB is enough.

Paths with a large BDP are called long fat networks (“long fat pipes”). They break assumptions in three ways: the 16-bit window (fixed by window scaling), the default socket buffer limits (fixed by tuning), and loss sensitivity. For a loss-based sender, throughput is roughly limited to (MSS / RTT) × 1.22 / √loss rate (the Mathis formula). At 200 ms and a loss rate of just 0.01%, that is about 7 Mbps, however big the window. On long paths, a tiny loss rate costs a huge amount of throughput; BBR, parallel connections or a server closer to the user are the cures.

#Head-of-line blocking, again

Because TCP delivers one ordered byte stream, a single lost segment holds back everything behind it, even data that has already arrived, until the retransmission fills the gap. Lesson 1.9 showed what this means for HTTP: HTTP/2 multiplexes many requests over one TCP connection, so one loss stalls all of them. That is TCP-level head-of-line blocking, and it is the main reason HTTP/3 runs on QUIC, which orders data per stream.

#Nagle’s algorithm, delayed ACK and TCP_NODELAY

Two optimisations from the 1980s save packets but can add latency when they meet.

• Nagle’s algorithm (sender): if there is unacknowledged data in flight, do not send a small (less than one MSS) segment; collect small writes until the ACK arrives or a full segment is ready. It was invented for terminals that sent one keystroke per packet: 1 byte of data in 41 bytes of packet.
• Delayed ACK (receiver): do not acknowledge every segment immediately. Wait a little (up to 40 ms on Linux, 200 ms on Windows; the standard allows up to 500 ms) hoping to piggyback the ACK on response data or to acknowledge two segments with one ACK. At least every second full segment is still acknowledged at once.

Each is sensible alone. Together, a write-write-read pattern stalls: the client writes a request header (sent at once), then the body (held by Nagle, waiting for the ACK of the header); the server has only half a request so it has nothing to reply with, and delays its ACK. Both wait until the delayed-ACK timer fires: 40 ms (or 200 ms) added to every request.

The fixes, best first:

1. Write each message in one call (build it in a buffer, or use writev), so there is never a small second write.
2. Set TCP_NODELAY on the socket to disable Nagle. Latency-sensitive software (most HTTP servers and clients, gRPC, many Redis and database client libraries, Go’s net package) does this by default. The cost is more small packets if the application writes many tiny pieces.
3. For bulk senders on Linux, TCP_CORK or MSG_MORE say “more data is coming, fill the segment first”; TCP_QUICKACK turns off delayed ACK temporarily on the receiver.

#TCP Fast Open and keepalive

TCP Fast Open (TFO) lets a returning client send data in the SYN. On the first connection the client asks for a cookie (a short code the server derives from the client’s IP address and a secret); on later connections it puts the cookie and its request in the SYN, and the server can answer in the SYN-ACK, saving one round trip. Two catches: SYN data can be replayed or duplicated, so only idempotent requests are safe; and some middleboxes drop SYNs carrying data, after which the client must fall back. Browsers never enabled TFO widely. Today the same saving comes from TLS 1.3 and QUIC 0-RTT, which run above the network’s interference.

Keepalive detects a peer that has disappeared without a FIN or RST (power cut, cable pulled, a NAT that forgot the mapping). An idle TCP connection sends nothing at all, so without keepalive it can sit “established” forever. When enabled per socket with SO_KEEPALIVE (it is off by default), Linux waits 2 hours idle, then sends up to 9 probes 75 seconds apart: a dead peer is detected only after 2 hours 11 minutes. Services lower this (for example 60 s idle, 10 s interval, 3 probes) or use application heartbeats (HTTP/2 ping frames, WebSocket pings, connection-pool validation), and set them shorter than any NAT or load balancer idle timeout on the path.

#TCP in system design

What all of this means when you design and run systems:

• New connections are slow; reuse them. A new connection pays one RTT for TCP, one or two for TLS, and then starts with only 10 segments of cwnd. Downloading 1 MB on a fresh connection takes about 7 round trips of slow start; on a warm connection with a large cwnd it takes 1–2. Use HTTP keep-alive, connection pools to databases and services, and HTTP/2 or HTTP/3 multiplexing.
• Keep warm connections warm. Linux resets cwnd after an idle period longer than the RTO (net.ipv4.tcp_slow_start_after_idle = 1). On servers with long-lived, bursty connections (APIs behind a load balancer, database pools), set it to 0.
• Cross-region throughput is limited by the BDP. Replicating a database or copying backups between continents needs large socket buffers (net.core.rmem_max, net.ipv4.tcp_rmem, tcp_wmem raised to 32–64 MB), window scaling intact through every firewall, and often several parallel streams (as aws s3 cp, rclone and multipart uploads do).
• Run BBR on servers that talk to distant or mobile clients. net.ipv4.tcp_congestion_control = bbr (with the fq queueing discipline) often raises throughput and lowers latency on lossy paths. Test fairness with your other traffic first; CUBIC remains the safe default inside a data centre.
• Short, chatty RPCs need TCP_NODELAY and single writes, or every call can gain 40 ms.
• Set application timeouts and heartbeats: TCP alone can wait 15 minutes for a dead server or 2 hours for a silent one.
• Zero windows in a capture point at the consumer, not the network: a slow client, an overloaded worker or a full disk.
• Connection storms hurt: thousands of clients reconnecting at once all start in slow start and all handshake at once. Spread reconnects with jitter (Chapter 28 covers retry and backoff).
• Why QUIC moved transport into user space: TCP lives in the operating system kernel, and its header is visible to (and often modified by) firewalls and NAT boxes, so changing it takes years (TFO and new options are still blocked by middleboxes). QUIC runs on UDP inside the application and encrypts its transport headers, so congestion control, loss recovery and new features ship with a browser or server update. It also removes TCP’s head-of-line blocking and the retransmission ambiguity (packet numbers are never reused).

#Try it yourself

# Linux: per-connection TCP internals (congestion algorithm, window scale, RTO, RTT, MSS, cwnd, retransmissions)
ss -tin state established '( dport = :443 )'
#   cubic wscale:7,7 rto:204 rtt:3.1/1.4 mss:1448 cwnd:10 ssthresh:20 bytes_acked:52011 retrans:0/2 ...

# Which congestion control is in use, and which are available?
sysctl net.ipv4.tcp_congestion_control
sysctl net.ipv4.tcp_available_congestion_control

# Buffer limits (min default max, in bytes) and other knobs from this lesson
sysctl net.ipv4.tcp_rmem net.ipv4.tcp_wmem net.ipv4.tcp_window_scaling net.ipv4.tcp_sack
sysctl net.ipv4.tcp_slow_start_after_idle net.ipv4.tcp_keepalive_time net.ipv4.tcp_fastopen

# Try BBR on a test machine (not production without testing)
sudo sysctl -w net.core.default_qdisc=fq
sudo sysctl -w net.ipv4.tcp_congestion_control=bbr

# Measure real throughput between two machines
iperf3 -s                         # on the server
iperf3 -c 203.0.113.50 -t 10      # one stream from the client
iperf3 -c 203.0.113.50 -P 4       # four parallel streams
iperf3 -c 203.0.113.50 -C bbr     # choose the congestion algorithm (Linux)

# Capture handshakes, resets and FINs only
sudo tcpdump -i any -nn 'tcp[tcpflags] & (tcp-syn|tcp-fin|tcp-rst) != 0'

# Is the port open? (connection refused = RST, a hang = filtered)
nc -vz example.com 443
telnet example.com 443

On Windows, netsh interface tcp show global shows auto-tuning and the congestion provider, and Get-NetTCPConnection in PowerShell lists connections and their states.

In Wireshark, open a capture of a large download and try these display filters: tcp.analysis.retransmission, tcp.analysis.fast_retransmission, tcp.analysis.duplicate_ack, tcp.analysis.zero_window, tcp.analysis.window_full, tcp.options.sack_le (segments carrying SACK blocks) and tcp.flags.reset == 1. Then use Statistics → TCP Stream Graphs → Stevens to see sequence numbers climb over time (slow start is the curve at the start), and Window Scaling to watch the receive window.

#Summary

Key takeaways

• The transport layer delivers process to process: ports, multiplexing, segmentation and reassembly, connection, error, flow and congestion control. TCP sends segments as a reliable, ordered, full-duplex byte stream; UDP sends independent datagrams.
• TCP header: 20–60 bytes: ports, 32-bit sequence (first byte) and acknowledgment (next byte expected) numbers, data offset (header length in 4-byte words), eight flags (CWR, ECE, URG, ACK, PSH, RST, SYN, FIN), 16-bit window, checksum, urgent pointer, options (MSS, window scale, SACK, timestamps). UDP header: 8 bytes: ports, length, checksum.
• Loss is repaired by the RTO (SRTT + 4 × RTTVAR, doubling on each timeout) or, faster, by fast retransmit after three duplicate ACKs; SACK repairs several holes in one RTT.
• Flow control: the receiver advertises rwnd; a full buffer gives a zero window, the sender sends window probes; window scaling lifts the 64 KB limit.
• Congestion control: the sender keeps cwnd and may send min(cwnd, rwnd). Slow start begins at 10 segments and doubles per RTT (one segment per ACK) up to ssthresh; then +1 segment per RTT; three duplicate ACKs halve it (AIMD sawtooth), a timeout drops it to one segment. CUBIC is the default today; BBR models bandwidth and RTT; ECN marks instead of dropping.
• BDP = bandwidth × RTT: 1 Gbps × 200 ms = 25 MB must be in flight. One connection’s throughput ≤ window / RTT, and on long paths even tiny loss rates cap loss-based TCP.
• Nagle + delayed ACK can add 40–200 ms per request; use single writes or TCP_NODELAY. TFO saves a round trip but is fragile; keepalive defaults detect a dead peer only after 2 h 11 min.
• In system design: reuse connections, tune buffers for cross-region transfers, consider BBR for distant clients, set application timeouts, and remember why QUIC moved transport into user space.

Self-check: TCP in depth31 questions