TCP, UDP, TLS and QUIC: Connections and Security

By the end of this lesson you will be able to

• Explain what the transport layer adds on top of IP, and choose between TCP and UDP for a given job.
• Walk through the TCP three-way handshake with real sequence numbers, and explain what happens when a SYN is lost, a port is closed, or a server is flooded.
• Explain how TCP makes delivery reliable: sequence numbers, ACKs, retransmission (timeout and fast retransmit), reordering, flow control and congestion control.
• Describe how a connection closes (FIN, TIME_WAIT, RST) and why reusing connections matters at scale.
• Explain head-of-line blocking and how HTTP/1.1, HTTP/2 and HTTP/3 differ.
• Walk through the TLS 1.3 handshake, say what a certificate proves, and recognise each certificate error.
• Explain what QUIC and HSTS add, and count the round trips each setup costs.

#Why we need a transport layer

After DNS gives the browser an IP address, the browser still has a problem. IP delivers packets from one machine to another, on a best-effort basis: packets can be lost, duplicated, delayed or arrive in a different order, and IP will not tell anyone. IP also knows only machines, not programs.

The transport layer sits on top of IP and solves both problems:

• Ports pick out the program. The server’s web server listens on port 443 (HTTPS) or 80 (HTTP); your browser uses a temporary (ephemeral) port such as 53000. The four values source IP, source port, destination IP, destination port identify one conversation.
• Delivery guarantees, if you want them. This is where the two main protocols differ:
  • TCP (Transmission Control Protocol) delivers data reliably and in order, keeping the sequence correct.
  • UDP (User Datagram Protocol) sends data fast but without a connection and without guarantees. It is also the foundation of QUIC.

On top of either one, TLS (Transport Layer Security) encrypts the traffic and proves who the server is. HTTPS is simply HTTP running inside TLS.

#TCP vs UDP

The rule of thumb: if every byte must arrive and in order (a web page, a bank transfer, a file), use TCP. If fresh data matters more than complete data (the position of a player in a game, the next 20 ms of a voice call), or the exchange is one tiny question and answer (DNS), use UDP.

#Opening a TCP connection

Once it has the server’s IP address, the browser either creates a new transport connection or reuses one it already has to that server. Reuse is common: a browser keeps connections open for a while precisely to avoid the setup cost you are about to see.

#The three-way handshake

For HTTPS over TCP, a new connection starts with three segments:

1. 1. SYN: client → server

   The client wants to start a connection, for example from client port 53000 to server port 443. It sets the SYN (synchronize) flag and picks a random Initial Sequence Number (ISN), say 1000.

2. 2. SYN-ACK: server → client

   The server accepts. It sets SYN (with its own ISN, say 5000) and ACK (acknowledgment) with ack=1001: “I received your SYN; next I expect byte 1001.” If the client is behind NAT, this reply actually arrives at the router’s public IP and mapped port, and the router forwards it inside.

3. 3. ACK: client → server

   The client confirms the server’s number with ack=5001. Both directions are now synchronized and the TCP connection is established. That cost one full round trip (RTT) before any data.

After this, an HTTPS connection usually performs the TLS handshake to set up security and encryption, and only then sends the HTTP request.

Why three messages and not two? Each side has to announce its own starting number and hear the other side confirm it. The server’s SYN and its ACK of the client’s SYN travel together in one segment, so four logical messages fit into three segments. Random ISNs make it hard for an attacker to guess the numbers and inject fake data into someone else’s connection.

During the handshake the two sides also agree options: the maximum segment size (MSS), window scaling (allowing large windows), and whether selective acknowledgment (SACK) is supported.

#When the handshake goes wrong

Play the other scenarios in the animation above:

• SYN lost. The client hears nothing, waits about 1 second, and resends. Each retry doubles the wait (1, 2, 4, 8 s…), which is called exponential backoff. After roughly five or six attempts the OS reports connection timed out. A page that “hangs for a few seconds, then loads” is often a lost SYN.
• Closed port. If the machine is up but nothing listens on that port, its OS answers the SYN with RST (reset). The application gets connection refused immediately. If instead a firewall silently drops the SYN, there is no RST, just retries and finally a timeout. So fast refusal means “host up, port closed”, and slow timeout means “filtered or unreachable”.
• SYN flood. An attacker sends huge numbers of SYNs from fake addresses and never sends the final ACK. Each one occupies a slot in the server’s queue of half-open connections, and real users are turned away: a denial of service (DoS), or DDoS when many machines join in. The classic defence is SYN cookies: the server stores nothing and instead encodes the connection details, with a secret hash, into its own ISN. Only a real client that received the SYN-ACK can return the right number. Large sites also absorb floods at the CDN or load balancer.

#Reliable delivery: sequence numbers and ACKs

TCP treats the data in each direction as one long stream of bytes and numbers every byte. Each segment carries the number of its first byte (seq); the receiver answers with the number of the next byte it expects (ack).

A worked example, continuing the handshake above (client ISN 1000, server ISN 5000):

Three details that interviewers like:

• ACKs are cumulative. ack=7001 confirms everything before 7001, so one ACK can cover several segments. Receivers often acknowledge every second segment (delayed ACK) to save packets.
• The sender does not wait for each ACK. It keeps several segments in flight at once, up to its window. Waiting for every ACK would make a 100 ms link crawl.
• SYN and FIN each count as one byte, which is why the first data byte is ISN + 1.

#When a segment is lost

TCP has two ways to notice a loss:

1. Retransmission timeout (RTO). When it sends data, the sender starts a timer, set from the measured round-trip time (a little over one RTT, at least 200 ms on Linux). If no ACK arrives in time, it resends. This is the slow path, and TCP reads it as serious congestion: the timeout doubles and the sending rate drops to the minimum.
2. Fast retransmit. If segment 2 is lost but segments 3, 4 and 5 arrive, the receiver answers each with the same ack=6001 (“I still want 6001”). These are duplicate ACKs. After three duplicate ACKs the sender concludes that one segment is missing while the network still works, and resends it immediately without waiting for the timer.

#Out-of-order delivery and reassembly

Packets can take different paths through the internet, so segment 3 may arrive before segment 2 even when nothing is lost. The receiver keeps out-of-order segments in its buffer, delivers to the application only the bytes that are in order, and hands up the rest as soon as the gap is filled. The application never sees the reordering. The flip side: while one segment is missing, everything behind it waits. Remember this; it is called head-of-line blocking.

#How the response gets back to your browser

Here is the full return path from the notes: the server replies, the home router translates the address, then TCP, TLS and the browser each do their part.

The router looks up port 62001 in its NAT mapping and sends the response to the right computer. On the computer, TCP delivers the data in the right order, TLS decrypts it (and checks it was not altered), and the browser processes the HTTP response and displays the page.

#Flow control and congestion control

A fast sender can overwhelm two different things: the receiver, and the network in between. TCP has a separate brake for each.

• Flow control (protects the receiver). Every ACK carries a receive window (rwnd): how many more bytes the receiver’s buffer can take right now. If the application reads slowly, the window shrinks, possibly to zero, and the sender must pause (sending only tiny window probes) until the window reopens.
• Congestion control (protects the network). The sender also keeps a congestion window (cwnd), its own guess of how much the path can carry. It may send at most the smaller of cwnd and rwnd.
  • Slow start: begin small (modern stacks: 10 segments) and add one segment per ACK, which doubles cwnd every round trip. Despite the name, growth is exponential.
  • Congestion avoidance: after reaching a threshold (ssthresh), grow by only one segment per round trip.
  • On loss: after three duplicate ACKs, halve cwnd and continue; after a timeout, drop back to the minimum and slow-start again.

The result is a “sawtooth”: speed climbs until a router’s queue overflows, halves, and climbs again. This is how millions of connections share the same links fairly. Modern algorithms such as CUBIC (the default on Linux, Windows and macOS) and Google’s BBR shape the curve differently, but the idea is the same.

Lesson 1.10 takes this apart in detail: every header field, how the timeout is computed, SACK, zero windows and window scaling, the exact slow-start and AIMD rules, CUBIC vs BBR, and the bandwidth-delay product.

#Closing a connection

• Graceful close (FIN). Each direction closes separately. The side that is done sends FIN; the other side ACKs it and may keep sending until it too sends FIN, which is then acknowledged: four segments (often three, when the second FIN rides with the ACK).
• TIME_WAIT. The side that closed first waits about a minute (twice the maximum segment lifetime; 60 s on Linux) before forgetting the connection. This lets it re-ACK a FIN whose ACK was lost, and stops delayed old packets from being mistaken for a new connection on the same ports.
• RST (abort). A reset tears the connection down instantly, without the FIN exchange and without TIME_WAIT; unsent data is lost. You see RST when a port is closed, when an application crashes or aborts, after a server reboot (packets for an unknown connection), and from some firewalls. Applications report it as connection reset by peer.

A busy client that opens and closes a connection for every request can pile up tens of thousands of sockets in TIME_WAIT and run out of local ports. The cure is the same as before: reuse connections.

#Head-of-line blocking and the HTTP versions

Head-of-line blocking means one slow or missing item holds up everything queued behind it, like a stalled car at the front of a single-lane road. It appears at two levels.

Keep-alive and connection reuse are the foundation of all three modern versions: the handshake cost is paid once and spread over many requests. HTTP/2 goes further and needs only one connection per site, which also means one congestion window that warms up properly. Its weakness shows on lossy networks (mobile, crowded Wi-Fi), where a single loss freezes the whole page; HTTP/3 was designed to fix exactly that.

#TLS: securing the connection

TLS (Transport Layer Security) encrypts network traffic to keep information safe, and helps verify the server’s identity. Its predecessor SSL is obsolete, though people still say “SSL certificate”. TLS gives three guarantees:

1. Confidentiality: nobody on the path can read the data (encryption).
2. Integrity: nobody can change the data without being detected (each record carries an integrity tag).
3. Authentication: you are really talking to www.example.com, not an impostor (certificates).

#Two kinds of cryptography

• Symmetric encryption uses one shared key to lock and unlock (for example AES). It is very fast, so all the actual data is encrypted this way. The problem: how do two strangers agree on a key over a network anyone can listen to?
• Asymmetric (public-key) cryptography uses a key pair: a public key anyone may know and a private key that never leaves its owner (for example RSA or elliptic-curve keys). It is much slower, so TLS uses it only during the handshake: to agree on a shared key (Diffie-Hellman key exchange, ECDHE) and to sign things (certificates, CertificateVerify).

#The TLS 1.3 handshake

1. 1. ClientHello (client → server)

   The client sends the TLS versions and cipher suites it supports, a random value, and already a key share (its half of the key exchange). Two important extensions: SNI (Server Name Indication), naming the site it wants so that a server hosting many sites picks the right certificate, and ALPN (Application-Layer Protocol Negotiation), offering h2 or http/1.1.

2. 2. ServerHello (server → client)

   The server picks the version and cipher suite and sends its own key share. Both sides can now compute the same shared secret, which never crossed the network. Everything after this point is encrypted.

3. 3. Certificate, CertificateVerify, Finished (server → client)

   The server sends its chosen ALPN protocol, its certificate chain, a CertificateVerify signature over the handshake made with the certificate’s private key (proof it owns the certificate), and Finished, a keyed hash of the whole handshake that detects any tampering.

4. 4. Finished + first request (client → server)

   The client checks the certificate, sends its own Finished and, in the same flight, the first HTTP request. TLS 1.3 costs one round trip (1-RTT); TLS 1.2 needed two.

#Session resumption and 0-RTT

After a handshake the server can give the client a session ticket. Next time, the client presents it as a pre-shared key (PSK) and the handshake skips the certificate. With 0-RTT, the client even sends its first request (early data) inside the very first flight, so no round trip is spent waiting.

The catch is replay: an attacker who recorded that first flight can send it again, and the server cannot tell. Harmless for GET /home, dangerous for POST /transfer-money. So servers accept 0-RTT only for safe, idempotent requests, or disable it.

#Certificates and the chain of trust

A certificate says: “this public key belongs to www.example.com, valid from date A to date B”, signed by a Certificate Authority (CA). Usually the site’s certificate (the leaf) is signed by an intermediate CA, which is signed by a root CA. Your OS or browser ships with a trust store of about 150 root CAs. Trust flows down the chain from a root you already have.

The browser checks:

1. Name: the hostname you typed is listed in the certificate (*.example.com covers www.example.com but not example.com or a.b.example.com).
2. Dates: today lies between “not before” and “not after”. Public certificates issued since 15 March 2026 last at most 200 days, falling to 100 days in 2027 and 47 days in 2029, so renewal must be automated (Let’s Encrypt, cloud certificate managers; see lesson 2.7).
3. Signature chain: each certificate is signed by the next, up to a trusted root.
4. Revocation: the certificate has not been cancelled early, checked via OCSP (often stapled by the server) or revocation lists (CRL).
5. Possession: CertificateVerify proves the server holds the private key.

In every case the browser stops before sending your request, cookies or password. A man-in-the-middle (MITM) on café Wi-Fi can relay your packets but cannot produce a valid certificate for someone else’s site, because it lacks that site’s private key. The attack only works if the user clicks through the warning.

#QUIC and HTTP/3

QUIC (originally Quick UDP Internet Connections) is a modern, fast and secure transport protocol built on UDP, and it is what HTTP/3 runs on. It reimplements what TCP does (acknowledgments, retransmission, flow and congestion control) and adds what TCP could not:

• TLS 1.3 built in. Transport and encryption are set up together: 1-RTT for a new connection, 0-RTT on resumption. Even most transport headers are encrypted.
• Independent streams. Data is ordered per stream, so a lost packet delays only its own stream: no TCP-level head-of-line blocking.
• Connection migration. A QUIC connection is identified by a connection ID, not by IP and port. When your phone moves from Wi-Fi to mobile data and its IP changes, the connection continues; a TCP connection would break and start over.
• Runs in user space. It ships inside browsers and servers, so it improves without waiting for OS updates.

Two practical caveats. Some networks (offices, hotels) block UDP port 443; browsers then fall back to TCP + TLS + HTTP/2, so every HTTP/3 server must also offer TCP. And browsers discover HTTP/3 support from an Alt-Svc response header or a DNS HTTPS record, so the very first visit often still uses TCP.

#The round-trip budget

Every round trip before the first request is pure waiting. On a 50 ms path (roughly Dhaka to Singapore):

DNS lookup time comes on top of all of these, and so does an HTTP-to-HTTPS redirect, which is one reason for HSTS.

#HSTS: forcing HTTPS

HSTS (HTTP Strict Transport Security) forces the browser to always use the secure HTTPS protocol for a site. A site sends it as a response header over HTTPS:

Strict-Transport-Security: max-age=31536000; includeSubDomains; preload

From then on, for max-age seconds (here one year), the browser rewrites any http:// link to https:// itself, before anything leaves your machine, and turns certificate errors into hard failures with no “proceed anyway” button.

The weakness is the first visit: before the browser has ever seen the header, typing example.com starts with plain HTTP, and an attacker on the network can keep you on HTTP (SSL stripping). The fix is the HSTS preload list: sites that qualify submit their domain, and browsers ship with the list built in, so even the first visit goes straight to HTTPS. Whole TLDs such as .dev and .app are preloaded.

#TCP, UDP and TLS in system design

• TLS termination. Load balancers and CDNs usually terminate TLS: they hold the certificate, decrypt, and talk to backends over a private network (or re-encrypt). This centralises certificates and moves crypto work off app servers. Terminating at a CDN edge near the user also makes handshakes faster, since the RTT is short.
• Connection pooling and keep-alive between services and to databases avoid handshakes, slow start and TIME_WAIT exhaustion.
• Long-lived connections (WebSockets, streaming APIs) need keep-alives to survive NAT timeouts, and load balancers must be configured with idle timeouts longer than the keep-alive interval.
• Picking the transport. Use TCP (or QUIC) for anything that must be complete. Use UDP for real-time media, game state and discovery, and accept that the application handles loss.
• Enable HTTP/2 and HTTP/3 at the edge; they help most for mobile users on lossy networks.
• Automate certificates and monitor expiry dates: an expired certificate is one of the most common and most avoidable outages.

#Try it yourself

# See the TCP connect, TLS handshake, certificate and HTTP exchange
curl -v https://example.com -o /dev/null
#   * Connected to example.com (...) port 443
#   * SSL connection using TLSv1.3 / ...      <- negotiated version and cipher
#   * ALPN: server accepted h2                <- HTTP/2 chosen through ALPN
#   * Server certificate: subject: CN=www.example.org ...

# Timing of each phase: DNS, TCP connect, TLS handshake, first byte
curl -so /dev/null -w "dns %{time_namelookup}  tcp %{time_connect}  tls %{time_appconnect}  ttfb %{time_starttransfer}\n" https://example.com

# Inspect the certificate chain and TLS details directly
openssl s_client -connect example.com:443 -servername example.com
# Try HTTP/3 (curl built with HTTP/3 support)
curl --http3 -I https://cloudflare.com

# List your open TCP connections and their states (ESTABLISHED, TIME_WAIT...)
netstat -an          # Windows, macOS, Linux
ss -tan              # Linux: TCP sockets with state
ss -uan              # Linux: UDP sockets

In Wireshark, capture while opening a site and use the display filter tcp.flags.syn==1 to see only handshakes (SYN and SYN-ACK). Use tcp.analysis.retransmission to find resent segments, tls.handshake for TLS handshakes, and quic for HTTP/3 traffic. Notice that after the ServerHello, TLS 1.3 shows only “Application Data”: even the certificate is encrypted.

#Summary

Key takeaways

• IP delivers packets on a best-effort basis to a machine; the transport layer adds ports (which program) and, with TCP, reliable ordered delivery.
• TCP: connection-oriented, reliable, ordered, with flow and congestion control. UDP: connectionless, 8-byte header, no guarantees, ideal for DNS, calls, games and as the base of QUIC.
• The three-way handshake (SYN, SYN-ACK, ACK) exchanges random ISNs and costs one RTT. A lost SYN is retried with backoff; a closed port answers RST; SYN floods are countered with SYN cookies.
• seq numbers bytes and ack names the next byte expected. Loss is repaired by timeout (slow) or fast retransmit after 3 duplicate ACKs; out-of-order data is buffered and reassembled.
• Flow control (receive window) protects the receiver; congestion control (slow start, avoidance, halving cwnd on loss) protects the network. New connections start slow, so reuse connections.
• Connections close with FIN (four segments, then TIME_WAIT on the closer) or abort with RST.
• Head-of-line blocking: HTTP/1.1 at the HTTP level, HTTP/2 at the TCP level, HTTP/3 over QUIC avoids both.
• TLS gives confidentiality, integrity and authentication. TLS 1.3 takes 1 RTT (ClientHello with key share, SNI, ALPN; ServerHello; Certificate, CertificateVerify, Finished), resumption allows 0-RTT with a replay risk. Browsers check name, dates, chain to a trusted root and revocation.
• QUIC = UDP + built-in TLS 1.3 + independent streams + connection migration; falls back to TCP when UDP is blocked. HSTS forces HTTPS; the preload list covers the first visit.

Self-check: TCP, UDP, TLS and QUIC30 questions