Real-Time II: WebSockets, WebRTC and Scaling Them

By the end of this lesson you will be able to

• Explain the WebSocket upgrade handshake header by header, including what Sec-WebSocket-Key and Sec-WebSocket-Accept prove and what they do not.
• Read a WebSocket frame (FIN, opcode, mask bit, length, masking key), say why its header is 2–14 bytes and why clients must mask.
• Use ping/pong heartbeats, close codes, subprotocols and wss:// correctly, and authenticate a socket without leaking tokens into logs or opening a cross-site hijack.
• Scale WebSockets: connections per server, load balancers and idle timeouts, stateless gateways with a pub/sub backplane, presence, delivery guarantees with acks and sequence numbers, reconnect storms, draining during deploys, and the fan-out maths for a million users.
• Explain how a WebRTC call is set up (signaling, SDP offer/answer, ICE, STUN, TURN), how it is encrypted (DTLS-SRTP), what data channels are, and when a group call needs an SFU or MCU.
• Choose between short polling, long polling, Server-Sent Events, WebSockets and WebRTC, and know where WebTransport, MQTT and gRPC streaming fit.

#Where this lesson fits

Lesson 2.5 covered the techniques that stay inside ordinary HTTP: short polling, long polling and Server-Sent Events. They all share one limit: the client cannot talk back on the same channel. This lesson covers the two technologies that remove that limit, and then the hard part of real-time systems: running them for millions of users.

Several pieces were introduced in Chapter 1, and this lesson builds on them instead of repeating them:

• Lesson 1.3 showed why a WebSocket server is stateful on purpose and sketched the pub/sub idea.
• Lesson 1.4 introduced WebRTC, signaling, STUN, TURN and the SFU; lesson 1.7 showed NAT, hole punching and symmetric NAT.
• Lesson 1.9 and lesson 1.10 covered TCP, TLS, keepalive and NAT idle timeouts; lesson 1.13 showed the Nginx settings a WebSocket proxy needs.

#WebSocket: one connection, both directions

A WebSocket is a full-duplex, bidirectional, persistent connection between a browser (or any client) and a server, standardised in RFC 6455 (2011). Full duplex means both sides can send at the same time; persistent means the connection stays open for minutes or hours, and either side can send a message whenever it wants, with no request in front of it. URLs use ws:// or, in practice always, wss://.

Client                                    Server
  |--- HTTP GET  (Upgrade: websocket) ----->|   handshake phase (plain HTTP/1.1)
  |<-- 101 Switching Protocols -------------|
  |=========================================|
  |          [ WebSocket connection ]       |   same TCP connection, no more HTTP
  |<======== frame (text or binary) =======>|
  |<======== frame (text or binary) =======>|

It works in three phases:

1. Handshake: the client sends a standard HTTP request with an Upgrade: websocket header.
2. Protocol switch: if the server agrees, it answers with status 101 Switching Protocols.
3. Data transfer: the connection leaves HTTP behind and speaks the WebSocket frame protocol. On this one connection, client and server can send each other messages at any moment.

Why does this matter? Compared with polling there are no empty round trips, no repeated HTTP headers (hundreds of bytes per request) and no delay waiting for the next poll: a message costs a few bytes of framing and arrives one network trip after it is sent. Text (UTF-8, usually JSON) and binary data (an ArrayBuffer or Blob in the browser, so Protobuf or raw audio) both travel natively.

#The upgrade handshake

The opening request is an ordinary HTTP/1.1 GET (it must be a GET, and at least HTTP/1.1):

GET /chat HTTP/1.1
Host: chat.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13
Origin: https://chat.example.com
Sec-WebSocket-Protocol: chat.v2, chat.v1
Sec-WebSocket-Extensions: permessage-deflate

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
Sec-WebSocket-Protocol: chat.v2

The key and accept pair proves that the server understood the WebSocket handshake, so a plain HTTP server or a cache replaying an old response cannot be mistaken for a WebSocket endpoint. It is not security: anyone can compute it. If the status is not 101, or the accept value is wrong, the browser fails the connection.

#ws:// and wss://

ws:// runs on port 80 in clear text; wss:// runs inside TLS, usually on port 443, exactly like HTTPS. Always use wss://:

• It encrypts the messages and authenticates the server.
• It gets through the internet far more reliably. Corporate proxies and mobile middleboxes that inspect plain HTTP often strip the Upgrade header or kill long connections; inside TLS they see only an encrypted stream.
• Browsers block ws:// from an HTTPS page (mixed content).

#Frames: the wire format

After the handshake, every message travels as one or more frames. The header is tiny:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-------+-+-------------+-------------------------------+
|F|R|R|R| opcode|M| Payload len |    Extended payload length    |
|I|S|S|S|  (4)  |A|     (7)     |   (16 bits if len = 126,      |
|N|V|V|V|       |S|             |    64 bits if len = 127)      |
| |1|2|3|       |K|             |                               |
+-+-+-+-+-------+-+-------------+ - - - - - - - - - - - - - - - +
|          ... extended payload length continued (64-bit case)  |
+-------------------------------+-------------------------------+
|        Masking key (4 bytes, client → server frames only)     |
+---------------------------------------------------------------+
|                         Payload data                          |
+---------------------------------------------------------------+

So the header is 2 to 14 bytes: 2 bytes for a small server frame, 4 with a 16-bit length, 10 with a 64-bit length, plus 4 for the mask on client frames. Server-to-client frames are 2–10 bytes; client-to-server frames 6–14 bytes.

Why clients must mask. Masking is not encryption: the key travels in the same frame. It protects intermediaries. Before WebSockets were standardised, researchers showed that a script could make the browser send bytes that a badly written transparent proxy would read as a fresh HTTP request for some other site, then cache the attacker’s “response” for everyone behind that proxy (cache poisoning). With a random mask chosen by the browser, the script can no longer control the bytes that appear on the wire. Servers do not need to mask because their bytes are not chosen by untrusted page scripts, and a server that receives an unmasked client frame must close the connection.

Messages vs frames. A message can be split into fragments: the first frame carries the opcode (text or binary) with FIN = 0, the following frames use opcode continuation, and the last has FIN = 1. This lets a sender stream a message of unknown size. Control frames (close, ping, pong) carry at most 125 bytes, are never fragmented and may be sent in between the fragments of a big message, so a heartbeat is never stuck behind a 50 MB upload. Unlike raw TCP, WebSocket keeps message boundaries: the receiver gets whole messages, not a byte stream.

#Subprotocols and extensions

WebSocket only moves messages; it says nothing about what they mean. A subprotocol, negotiated with Sec-WebSocket-Protocol, names the application protocol on top: for example graphql-transport-ws for GraphQL subscriptions, mqtt for MQTT over WebSockets, or STOMP for message brokers. Version your own protocol the same way (chat.v2), so old and new clients can be served side by side during a migration. If the server picks a subprotocol the client did not offer, the client fails the connection.

The only common extension is permessage-deflate (compression). It can shrink JSON by 70–90%, but it costs memory per connection (a compression context on each side, tens to hundreds of kilobytes, unless you disable context takeover) and CPU, and compressing secrets together with attacker-chosen text can leak them (the same class of attack as CRIME/BREACH in HTTPS). Many large deployments enable it only for big messages, or not at all.

#Closing: close frames and status codes

A clean close is a closing handshake: one side sends a close frame with a 2-byte status code and an optional reason (at most 123 bytes), the other answers with a close frame, then the TCP connection closes (normally by the server first). Codes worth knowing:

Clients should decide what to do from the code: 1000 means do not reconnect, 1012 means reconnect at once (with jitter), 1013 means back off, 1006 means the network failed, so reconnect with backoff and resume.

#Heartbeats: ping and pong

A WebSocket that carries no data sends nothing at all, and that causes two problems:

1. Middleboxes cut quiet connections. Load balancers, proxies, NAT gateways and mobile carriers drop connections that have been idle for a while: an AWS ALB after 60 seconds by default (configurable up to 4,000 s), an NLB after 350 seconds, Nginx’s proxy_read_timeout after 60 seconds, and home routers and carriers after a few minutes.
2. Dead peers look alive. A phone that walks into a lift or loses its network sends no FIN and no close frame. The server keeps a half-open connection: writes succeed (they only fill a kernel buffer), presence says “online”, and the socket leaks memory. TCP keepalive is off by default and, when on, takes over 2 hours to notice (lesson 1.10).

The fix is a heartbeat: the server sends a ping frame every 20–30 seconds; the other side must answer with a pong carrying the same payload. Two missed pongs, and the server closes the socket and cleans up. The interval must be shorter than the shortest idle timeout on the path. Browsers answer pings automatically, but JavaScript cannot send a ping frame, so clients that want to detect a dead server send a small application message ({"t":"ping"}) and reconnect if no reply arrives within about 10 seconds.

#Authenticating a WebSocket

The browser WebSocket API cannot set custom headers such as Authorization. The usual options:

Non-browser clients (mobile apps, services) can simply send an Authorization header on the upgrade. Whatever the method, remember that a socket can outlive the token that opened it: check expiry, and close with an application code (such as 4001) or accept a refreshed token over the socket when it runs out. Authorise every message (may this user post to this room?), not only the connection.

#WebSockets over HTTP/2 and HTTP/3

The classic upgrade needs HTTP/1.1 and a TCP connection of its own. RFC 8441 (2018) defines WebSockets over HTTP/2: the client sends an extended CONNECT request with :protocol: websocket on a new stream, the server answers 200 instead of 101, and the WebSocket then runs as one stream of an existing HTTP/2 connection, so many sockets to the same server share one TCP and TLS connection. RFC 9220 (2022) does the same for HTTP/3. Chrome and Firefox support RFC 8441, but many proxies and servers still accept only the HTTP/1.1 upgrade, so in practice most WebSocket traffic in 2026 still uses its own TCP connection. The frame format is identical either way.

#Scaling WebSockets

A request-response API is easy to scale because servers forget each request once it is answered. A WebSocket server holds long-lived, stateful connections: which user is on which socket, which rooms they joined. That changes almost every part of the design.

#How many connections can one server hold?

Each connection costs:

• A file descriptor. Linux processes often start with a soft limit of 1,024 open files (ulimit -n). Raise it (LimitNOFILE in systemd, nofile in limits.conf, and fs.nr_open/fs.file-max for the system) to a million or more.
• Memory. The kernel needs a few kilobytes per idle TCP socket (buffers grow only when data flows); TLS state, read and write buffers and per-connection objects in your runtime usually add 10–50 KB. Budget conservatively: 100,000 sockets × 50 KB = 5 GB.
• CPU, mostly at the edges. An idle socket costs almost nothing. TLS handshakes, heartbeats, JSON encoding and, above all, the messages you push are what use the CPU.

What makes huge numbers possible is the event loop: one thread asks the kernel (with epoll on Linux, kqueue on BSD) “which of my 100,000 sockets have data?” and handles only those, instead of one blocked thread per connection. Nginx, Node.js, Netty, Go’s runtime and Erlang all work this way (I/O models are covered in Chapter 3). The records are impressive: WhatsApp ran about 2 million connections per server on Erlang and FreeBSD back in 2012, and the Phoenix framework demonstrated 2 million on one machine. In production you deliberately hold far fewer, typically 10,000–100,000 per node, because a node that dies takes all its sockets with it (the blast radius), and those clients all reconnect at once.

#Load balancers and long-lived connections

• L4 (an NLB, or HAProxy in TCP mode) forwards TCP connections without reading them. It is cheap and handles huge connection counts, but cannot route by path or check the upgrade.
• L7 (an ALB, Nginx, Envoy, HAProxy in HTTP mode) sees the upgrade request, so it can terminate TLS, route /ws to the socket servers, and reject bad requests early. After the 101 it simply copies bytes both ways. It holds two connections per client (one to the client, one to the backend), so its own file descriptors and memory matter too.
• The proxy’s port limit. Every backend connection from a proxy uses an ephemeral port: Linux’s default range 32768–60999 gives about 28,000 connections per proxy IP to one backend IP and port. At high socket counts, give the backends several IPs or ports, or give the proxy several source IPs.
• Idle timeouts on every hop must be longer than your heartbeat interval (see above).
• Sticky sessions? A WebSocket is already pinned to one backend for its whole life, so the socket itself needs no stickiness. Stickiness only matters for fallbacks made of separate HTTP requests (Socket.IO’s long polling) or if a reconnect must return to the same server, which a good design avoids.
• Imbalance. Round robin balances new connections, not existing ones. After a scale-out the new nodes sit empty while old ones stay full, because nobody reconnects. Use least-connections, and rebalance gradually by closing a small share of sockets on the busiest nodes (code 1012) so they reconnect elsewhere.

#Stateless gateways and a pub/sub backplane

With one server, delivering a message to user B is a lookup in a local map. With two, user B may be connected to the other server, and the first one has no way to reach that socket (lesson 1.3 showed the problem). The standard architecture:

1. Gateways (connection servers) terminate WebSockets, authenticate, run heartbeats and translate between frames and internal messages. They hold sockets but no business state: any user may connect to any gateway, and losing a gateway loses no data.
2. Services behind them (chat, notifications, game logic) do the work and store anything that matters before delivering it.
3. A pub/sub backplane connects the two. Each gateway subscribes to a channel for each of its connected users (user:B) and for each room its users joined (room:r7). To deliver, a service publishes once to the channel; only the gateways subscribed to it receive a copy and write it to their local sockets.
4. Optionally a connection registry (user → gateway, in Redis with a TTL refreshed by heartbeats) for direct routing and presence.

For a group, the backplane sends one copy per gateway that has a member, not one per member; each gateway fans out locally. A 10,000-member room spread over 100 gateways costs the backplane 100 deliveries per message.

A common split: Kafka (or the database) is the durable source of truth, and a light pub/sub (Redis or NATS) does the fast, lossy fan-out to gateways, with gaps filled from storage on resume.

#Presence

“Who is online” looks simple and is expensive. A typical design: each gateway refreshes presence:user123 = gw-7 with a 60-second TTL on every heartbeat; if the key expires, the user is offline. Two problems:

• Flapping. Mobile connections drop and come back constantly. Wait a few seconds (a grace period) before announcing “offline”, or every tunnel turns into an offline–online burst.
• Fan-out. If each of 1 million users has 200 contacts and 100,000 status changes happen per minute, pushing every change to every contact is 20 million events per minute. Real systems push presence only to users who are looking (an open chat, a visible contact list) and fetch the rest on demand; very large groups show only a count.

#Delivery guarantees and ordering

TCP guarantees order and delivery only while the connection lives. When it breaks, bytes in flight or in a kernel buffer are lost, and neither side knows which ones arrived. Over a single socket, delivery is at most once. Chat-grade reliability is rebuilt on top:

1. Store before deliver. Write the message to durable storage first, with a sequence number per conversation assigned by a single writer (the database, or the partition that owns the conversation).
2. Acknowledge. The client acks the highest sequence number it has saved (cumulatively, like TCP).
3. Resume. On reconnect, to any gateway, the client says “I have everything up to 42”, and the server replays 43 onwards from storage.
4. De-duplicate. Replays make delivery at least once, so the client ignores sequence numbers it already has. For messages it sends, the client attaches a unique ID and retries with the same ID until acked; the server returns the original result for an ID it has seen (an idempotency key, Chapter 26).
5. Order by sequence number, not arrival. Messages can overtake each other across gateways and retries. The client buffers out-of-order messages, detects gaps (46, 48: where is 47?) and fetches what is missing.

At-least-once delivery plus idempotent processing gives the user the effect of exactly once. Ordering is guaranteed per conversation; nothing guarantees a global order across conversations, and nothing needs one.

#Reconnect storms, backoff with jitter, and draining

When a gateway with 50,000 sockets dies, 50,000 clients notice within the same second. If they all reconnect immediately, the other gateways face 50,000 TLS handshakes, the auth service 50,000 token checks, and the store 50,000 resume queries in one second. Many fail, and if every client retries after a fixed delay, the next wave is just as big: a thundering herd that can take down the healthy part of the fleet.

• Exponential backoff with full jitter. Wait random(0, min(cap, base × 2^attempt)), for example base 1 s and cap 30 s, and randomise the very first attempt too. 50,000 reconnects spread over 30 seconds are about 1,700 per second.
• Server-side admission control. Cap new handshakes per second per gateway; above it, answer cheaply with HTTP 503 and Retry-After before the upgrade, or close code 1013 on an open socket. Clients treat the hint as a minimum and add jitter (retries and backoff in depth: Chapter 28).
• Make reconnects cheap. Resume tokens, cached auth decisions and a resume query that reads only the gap.

Deploys are planned reconnect storms, so do them gently. Mark the gateway as draining (the load balancer stops sending new connections; an ALB’s deregistration delay defaults to 300 seconds), close its sockets in batches with code 1012 over several minutes, and stop the process only when it is empty. On Kubernetes the default 30-second termination grace period is far too short: raise it to cover the drain window, drain in a preStop hook, and roll one gateway at a time.

#Fan-out maths: one million concurrent users

A worked estimate for a chat service:

1. Connections. 1,000,000 concurrent users at a policy of 10,000 sockets per gateway = 100 gateways; with 30% headroom for failures and deploys, about 130. (Each could hold more; 10,000 keeps the blast radius and reconnect storm of one failure small.)
2. Heartbeats. One ping every 30 seconds = 1,000,000 / 30 ≈ 33,000 pings per second fleet-wide, about 330 per gateway: negligible.
3. Messages. If an average user receives one message every 10 seconds, that is 100,000 deliveries per second, 1,000 per gateway. At 300 bytes each, 30 MB/s or about 240 Mbps of outbound traffic in total. At a 5× peak (a big match ends), 500,000 per second.
4. Groups. A 100,000-member live-event room with 20 messages per second would mean 2 million deliveries per second from that one room. The backplane carries only 20 × 100 gateways = 2,000 per second; the gateways do the multiplication locally. At this size you also sample or batch (show “and 300 more messages”), because no human reads 20 messages a second.
5. Reconnects. Losing one gateway = 10,000 reconnects, about 330 per second over a 30-second jitter window. Losing a whole zone (a third of the fleet) = 330,000, which is why handshake capacity, not steady-state load, often sizes the fleet.

#Managed and mobile options

• Managed WebSockets. Amazon API Gateway WebSocket APIs hold the connections for you and call your backend (for example Lambda) per message; you push back through a callback URL (@connections). Limits to know: a connection lives at most 2 hours and is closed after 10 idle minutes. Hosted services such as Pusher and Ably offer the whole fan-out layer as a product.
• Mobile apps in the background cannot keep a socket open: the operating system suspends them. Chat apps keep a socket while the app is in front and use push notifications (APNs, FCM) to wake the user otherwise.

#WebRTC: real-time media between peers

WebRTC (Web Real-Time Communication) is a set of browser APIs (a W3C standard since 2021) and IETF protocols for sending audio, video and arbitrary data directly between two browsers or apps, peer to peer. It is what runs a Google Meet call in a browser tab without a plug-in.

“Directly” needs a correction: WebRTC sends media peer to peer, but it cannot start a call without servers. It needs a signaling server to introduce the peers, STUN to discover public addresses and usually TURN as a relay of last resort, and group calls send media through a server too.

               +-------------------+
               |  Signaling server |   (setup only: SDP and ICE candidates)
               +-------------------+
                  /             \
       1. SDP / ICE             2. SDP / ICE
                /                 \
               v                   v
        +----------+           +----------+
        |  Peer A  |<=========>|  Peer B  |
        +----------+           +----------+
          3. Direct encrypted media (SRTP over UDP), or via TURN if direct fails

#Signaling: SDP offer and answer

WebRTC deliberately does not standardise signaling: how the two peers exchange their setup messages is up to the application. Most apps use their own WebSocket (which is why the two technologies so often appear together); HTTP works too. Live-streaming ingest now has a standard, WHIP (RFC 9725, 2025): one HTTP POST of the offer to a media server.

What is exchanged is SDP (Session Description Protocol), a text description of the session:

1. The caller creates an offer: its media tracks, the codecs it supports (Opus for audio; VP8, H.264, and often VP9 or AV1 for video), ICE username and password, and the fingerprint of its DTLS certificate.
2. The callee applies it and creates an answer, choosing codecs both support and adding its own ICE credentials and fingerprint.
3. Each side then sends ICE candidates (possible addresses) as it discovers them (trickle ICE), again through signaling.

The signaling channel must stay up during the call, because a network change needs a new offer and answer (an ICE restart).

#ICE, STUN and TURN

Most devices sit behind NAT and have no public address (lesson 1.7). ICE (Interactive Connectivity Establishment) finds a path that works:

Both peers send connectivity checks (small STUN packets) on candidate pairs at the same time; each outgoing check opens a mapping in the sender’s own NAT, so the other side’s checks can get in (hole punching). ICE nominates the best pair that works, preferring host, then server-reflexive, then relay. While the call runs, consent checks every few seconds confirm the path still works; if they stop, the connection goes to “disconnected” and then “failed”, and the application must trigger an ICE restart.

STUN is cheap: two tiny packets per lookup, and public STUN servers exist. TURN is expensive, because it relays the whole media stream:

• Hole punching fails with a symmetric NAT (a new port for every destination), common in corporate networks and some mobile carriers, and when UDP is blocked entirely. Operators commonly report that roughly 10–20% of sessions need TURN, far more on corporate networks; the share depends heavily on your users.
• A relayed one-to-one call at 1.5 Mbps each way puts about 3 Mbps into and 3 Mbps out of the relay, roughly 1.35 GB of egress per call-hour. At cloud egress prices that is a real line on the bill.
• Run TURN on UDP 3478, and also over TLS on port 443 (turns:) for networks that allow only web traffic. Media inside TCP suffers head-of-line blocking, so this is the last resort.
• TURN needs short-lived credentials issued by your app (typically an HMAC of an expiry time), or it becomes a free open relay for strangers.

#Encryption: DTLS-SRTP

Encryption in WebRTC is mandatory; there is no unencrypted mode. Once ICE has found a path, the peers run a DTLS handshake (TLS adapted for UDP) over it, each using a self-signed certificate. Each side checks that the other’s certificate matches the fingerprint it received in the SDP, then both derive keys for SRTP (Secure RTP), which encrypts and authenticates every audio and video packet. This is DTLS-SRTP (RFC 5764).

Two consequences for design: the call is only as trustworthy as the signaling channel that carried the fingerprints (so signaling must run over TLS and be authenticated), and a TURN relay sees only encrypted packets. An SFU, however, terminates DTLS-SRTP and can see the media; true end-to-end encryption through an SFU needs an extra layer such as SFrame (RFC 9605), which encrypts the media payload before it reaches the SFU.

#Data channels

Besides media, WebRTC can open data channels for arbitrary messages between peers. They run SCTP over DTLS over UDP (RFC 8831): DTLS provides the security, SCTP the messages and streams. Each channel can be configured:

• Reliable and ordered (the default), like a TCP stream but per channel, so one slow channel does not block the others.
• Unordered and/or unreliable (ordered: false, maxRetransmits: 0 or maxPacketLifeTime), ideal for game state, cursor positions or sensor readings, where a newer value makes a lost older one worthless.

Uses: peer-to-peer file transfer (Snapdrop-style local sharing; Snapdrop itself changed owners in 2023 and is continued by a community fork), multiplayer games, remote control, and chat inside a call. Keep messages small (tens of kilobytes) and chunk large files.

#Group calls: mesh, SFU and MCU

One-to-one calls are naturally peer to peer. Group calls are not:

With 6 people at 1.5 Mbps each, a mesh makes everyone upload 7.5 Mbps and encode up to five times, while an SFU needs only 1.5 Mbps up per person (the SFU itself takes in 9 Mbps and sends out 45 Mbps).

SFUs use two tricks to serve receivers with different screens and networks:

• Simulcast: each sender encodes the same video at two or three resolutions (say 720p, 360p, 180p) and the SFU forwards the right one to each receiver: full size for the active speaker, thumbnails for the grid, low quality for a weak mobile link.
• SVC (Scalable Video Coding, supported by VP9 and AV1): one stream built in layers; the SFU drops layers instead of choosing between copies.

Large meetings cascade several SFUs across regions, and very large audiences (tens of thousands of viewers) switch from interactive WebRTC to broadcast-style delivery (HLS or DASH via a CDN), unless sub-second delay is essential.

Real products: Google Meet, Discord, Microsoft Teams and most video platforms run WebRTC media through SFUs. Zoom’s desktop and mobile apps use Zoom’s own media protocol, and its browser client uses WebRTC; either way, Zoom calls go through its servers, not peer to peer.

#Choosing a real-time technique

The five options side by side (corrected from the notes, see below):

How to choose:

                    What flows, and between whom?
                                 |
     +---------------------------+---------------------------+
     |                           |                           |
Server → client only    Client ⇄ server, both ways     Peer ⇄ peer media
     |                           |                           |
  (use SSE)               (use WebSocket)               (use WebRTC)
  • live stock ticker     • chat applications           • video calls (Meet)
  • news feed updates     • multiplayer games           • voice chat (Discord)
  • sports scorecard      • collaborative canvas        • P2P file sharing
  • AI output streaming     (Figma, Google Docs)
    (ChatGPT-style text)  • live trading screens

1. Short polling: when updates are rare and not urgent, and you want no extra moving parts. Example: a widget that changes once or twice a day, a status check.
2. Long polling: as a fallback where WebSockets or SSE do not get through (old browsers, hostile proxies). Example: an old payment gateway’s notification check.
3. Server-Sent Events: when data flows only server to client and the client rarely sends anything back. Examples: ChatGPT-style AI response streaming, live stock prices, live scoreboards, system notifications.
4. WebSockets: when both sides send often and instantly. Examples: live chat, multiplayer online games, live trading platforms, Figma-style collaborative editing.
5. WebRTC: when two or more people exchange audio, video or raw data with the lowest delay. Examples: Google Meet, Discord voice and video, browser-to-browser file sharing.

Rules of thumb for interviews: start with the simplest thing that meets the latency need; SSE covers a surprising share of “real-time” features and keeps plain HTTP semantics; choose WebSockets when the client really talks back often; choose WebRTC only for media or truly peer-to-peer data. Many products mix them: a chat app might use WebSockets for messages, push notifications in the background, and WebRTC for calls, with the WebSocket doubling as the call’s signaling channel.

#Newer and adjacent options

• WebTransport is a browser API over HTTP/3 (QUIC): many independent streams with no head-of-line blocking between them, plus unreliable datagrams, client to server. Think of it as WebSocket without TCP’s limits, for games, live media (the IETF’s Media over QUIC work builds on it) and high-rate telemetry. Chromium browsers and Firefox support it; Safari support has lagged, and it needs UDP to be open, so keep a WebSocket fallback.
• MQTT is the publish/subscribe protocol of the IoT world: devices keep one TCP connection to a broker (port 1883, or 8883 with TLS), publish to topics and subscribe with wildcards. It offers three delivery levels (QoS 0, 1, 2), retained messages (a new subscriber gets the last value) and a “last will” published if a device disappears. Its fixed header can be as small as 2 bytes, which suits battery-powered devices on flaky links. Browsers reach brokers through MQTT over WebSockets. Examples: the open-source Eclipse Mosquitto broker and managed AWS IoT Core.
• gRPC streaming (server, client and bidirectional streams over HTTP/2) is the usual choice for real-time streams between services; browsers need gRPC-Web, which supports only server streaming. It is covered in Chapter 26.
• Webhooks are server-to-server push over plain HTTP requests (Chapter 26), and mobile push (APNs, FCM) reaches apps that are not running.

#Real-time in system design

A checklist for any design with WebSockets:

• Separate connection handling from business logic: stateless gateways, services behind them, a pub/sub backplane between.
• Store before you deliver, number messages per conversation, ack, resume, de-duplicate.
• Heartbeat every 20–30 seconds; set every idle timeout on the path longer than that.
• Authenticate the upgrade without long-lived tokens in URLs; check Origin; re-check authorisation per message and when tokens expire.
• Limit everything per connection: maximum message size, messages per second, send-buffer size. A slow consumer whose buffer keeps growing (bufferedAmount in the browser, unsent bytes on the server) must be dropped or downgraded, or it will eat the server’s memory.
• Plan for reconnect storms: full-jitter backoff in clients, admission control in gateways, gradual draining in deploys, and capacity sized for losing a zone.
• Measure: open connections per gateway, connects and disconnects per second (by close code), message lag from store to socket, heartbeat failures, send-buffer sizes.

Interview angles to prepare: “Design WhatsApp” or “design a notification system” (gateways, backplane, store-and-forward, delivery receipts, presence, push for offline users; the full case study is in Chapter 52), “how do you scale to 10 million connections?” (connection maths, blast radius, storms), “SSE or WebSocket for streaming an LLM’s answer?” (SSE: one direction, plain HTTP, automatic resume), and “why is a group video call not peer to peer?” (upload bandwidth, SFU).

#Try it yourself

# 1. See a real handshake: send the upgrade request by hand (any WebSocket endpoint you control)
curl -i -N --http1.1 \
  -H "Connection: Upgrade" -H "Upgrade: websocket" \
  -H "Sec-WebSocket-Version: 13" \
  -H "Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==" \
  https://chat.example.com/ws
# Expect: HTTP/1.1 101 Switching Protocols ... Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

# 2. Compute the accept value yourself
printf '%s' 'dGhlIHNhbXBsZSBub25jZQ==258EAFA5-E914-47DA-95CA-C5AB0DC85B11' | openssl sha1 -binary | base64

# 3. Talk to a socket interactively (Node.js needed)
npx wscat -c wss://chat.example.com/ws

# 4. Connection capacity on a Linux server
ulimit -n                         # per-process open-file limit (often 1024)
sysctl fs.file-max fs.nr_open     # system-wide limits
ss -s                             # socket totals
ss -tn state established '( sport = :443 )' | wc -l   # established connections on 443

A minimal server with heartbeats, using the popular ws library for Node.js:

import { WebSocketServer } from 'ws';

const wss = new WebSocketServer({ port: 8080, maxPayload: 64 * 1024 }); // 1009 above 64 KB

wss.on('connection', (ws, req) => {
  if (req.headers.origin !== 'https://chat.example.com') return ws.close(1008, 'bad origin');
  ws.isAlive = true;
  ws.on('pong', () => { ws.isAlive = true; });
  ws.on('message', (data, isBinary) => { /* authorise, store, then publish to the backplane */ });
});

setInterval(() => {               // heartbeat: ping every 30 s, drop sockets that missed one
  for (const ws of wss.clients) {
    if (!ws.isAlive) { ws.terminate(); continue; }
    ws.isAlive = false;
    ws.ping();
  }
}, 30_000);

A browser client with resume and full-jitter backoff:

let attempt = 0;
let lastSeq = Number(localStorage.getItem('lastSeq') ?? 0);

function connect(ticket) {
  const ws = new WebSocket(`wss://chat.example.com/ws?ticket=${ticket}`); // one-time ticket only
  ws.onopen = () => { attempt = 0; ws.send(JSON.stringify({ type: 'resume', after: lastSeq })); };
  ws.onmessage = (e) => {
    const msg = JSON.parse(e.data);
    if (msg.seq <= lastSeq) return;            // duplicate from a replay
    lastSeq = msg.seq;                          // (real code also buffers gaps)
    ws.send(JSON.stringify({ type: 'ack', seq: lastSeq }));
  };
  ws.onclose = (e) => {
    if (e.code === 1000) return;               // closed on purpose
    const cap = Math.min(30_000, 1_000 * 2 ** attempt++);
    setTimeout(() => getTicket().then(connect), Math.random() * cap); // full jitter
  };
}

The Nginx proxy settings from lesson 1.13, with a timeout longer than the heartbeat:

location /ws {
    proxy_pass http://gateways;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_read_timeout 75s;   # must exceed the 30 s ping interval
}

In the browser, open DevTools → Network → filter WS, click a socket and open the Messages tab to watch frames, pings excluded. For WebRTC, start a call and open chrome://webrtc-internals (or about:webrtc in Firefox): you can see the offer and answer SDP, every ICE candidate with its type (host, srflx, relay), the selected pair, and live bitrates and packet loss.

#Summary

Key takeaways

• A WebSocket is a full-duplex, persistent connection that starts as an HTTP/1.1 GET with Upgrade: websocket and becomes frames after 101 Switching Protocols. Sec-WebSocket-Accept = base64(SHA-1(key + fixed string)) proves the server speaks WebSocket; it is not security. Always use wss://.
• Frames have a 2–14 byte header: FIN, RSV, opcode (text, binary, continuation, close, ping, pong), mask bit, 7/16/64-bit length, and a 4-byte masking key on every client frame, which protects proxies from cache poisoning, not your data. Messages can be fragmented; control frames are ≤ 125 bytes and can interleave.
• Heartbeats (ping/pong every 20–30 s) beat idle timeouts and detect half-open connections. Close codes tell clients whether to reconnect (1000 no, 1001/1012 yes, 1013 later, 1006 = connection dropped).
• Authenticate with cookies plus an Origin check (against CSWSH), a token in the first message, or a one-time ticket; never a long-lived token in the URL. Re-check authorisation over the socket’s lifetime.
• Scale with stateless gateways and a pub/sub backplane (Redis, NATS, Kafka): publish once per channel, one copy per gateway, local fan-out. Memory and file descriptors bound sockets per server; keep 10k–100k per node for a small blast radius.
• Delivery over one socket is at most once: store first, sequence numbers per conversation, acks, resume, client IDs for idempotent sends; order by sequence, not arrival.
• Avoid reconnect storms with full-jitter backoff and admission control; drain gateways gradually on deploys (close 1012 in batches, long grace periods).
• WebRTC sends media peer to peer but needs signaling (not standardised, often a WebSocket) for the SDP offer/answer, ICE with STUN for hole punching, and TURN to relay when NAT or firewalls block it (about 10–20% of sessions, and it costs bandwidth). Media is always encrypted with DTLS-SRTP; data channels are SCTP over DTLS.
• Group calls use an SFU (with simulcast or SVC), rarely an MCU; a mesh only works for a handful of people.
• Choose the simplest tool: polling for rare updates, SSE for server-to-client streams, WebSocket for two-way messaging, WebRTC for media; WebTransport is the HTTP/3 newcomer, MQTT the IoT standard, and gRPC streaming the service-to-service option.

Self-check: WebSockets, WebRTC and scaling them33 questions