Real-Time I: Short Polling, Long Polling and SSE

By the end of this lesson you will be able to

• Explain why plain HTTP cannot push, and name every way a server can still tell a client that something changed: short polling, long polling, server-sent events, WebSockets, WebRTC, webhooks and mobile push.
• Calculate the cost of short polling (requests per second, share of empty answers, average and worst delay) and make it cheaper with conditional requests, back-off and caching.
• Build long polling correctly: hold times below every timeout on the path, a cursor so no event falls into the gap, one poll in flight, and jittered retries.
• Read and write the text/event-stream format (data, event, id, retry, comments) and use EventSource, including automatic reconnection with Last-Event-ID.
• Run SSE in production: proxy buffering, compression, idle timeouts and heartbeats, the six-connection limit of HTTP/1.1, authentication without custom headers, deploys and reconnect storms.
• Explain why LLM chat APIs stream tokens over SSE, and design webhooks (retries, signatures, idempotency) and mobile push (APNs, FCM) as the push channels outside the browser tab.
• Choose between the three techniques with numbers, and defend the choice in a design interview.

#Where this lesson fits

Lesson 1.1 set the rule of the web: the client asks, the server answers. A server never speaks first. That is fine for loading a page, and a problem for everything that changes while the page is open: a chat message, the rider picking up your food, a cricket score, a build log, the next words of an AI answer.

Web applications use five main techniques to keep a client up to date beyond plain request-response:

This lesson covers the first three, which all work with ordinary HTTP, plus the two push channels that live outside the browser tab: webhooks (server to server) and mobile push (APNs and FCM). Lesson 2.6 covers WebSockets and WebRTC, compares all five side by side, and shows how to scale millions of open connections with pub/sub.

You will lean on a few earlier lessons: keep-alive connections and idle timeouts (1.10), NAT devices that forget idle mappings (1.7), reverse proxies, buffering and proxy_read_timeout (1.13), stateful connections and sticky sessions (1.3), and ETags with 304 Not Modified (2.4).

#The problem: HTTP only answers

Why can the server not just send a message to the browser when something happens?

• HTTP is request-response. A response is only allowed as the answer to a request. With no request waiting, there is nothing to answer.
• The browser is not reachable. It sits behind home NAT, carrier-grade NAT, firewalls and Wi-Fi that change every few minutes (lesson 1.7). It has no public address and does not listen on a port. Every connection must be opened from the client outwards.
• So every technique is a trick to get a request waiting on the server, or a connection open from the client, at the moment news appears. They differ in how often the client asks, how long a request or connection stays open, and what happens when it breaks.

Any push design has to answer three questions, and you will see them in every section:

1. Latency: how long between the event and the user seeing it?
2. Cost: how many requests, open connections, bytes and server memory does it take, per client and for the whole fleet?
3. Reliability: what happens to events while the client is disconnected, and can they arrive twice or out of order?

#Short polling

Short polling is the oldest and simplest technique. The client sends an ordinary HTTP request on a timer (every 3 or 5 seconds, say) to ask whether there is anything new. The server answers immediately, whether there is news or not.

Client                                  Server
  |--- GET /orders/42/status ----------->|
  |<-- 200 "preparing" (nothing new) ----|
  |        (wait 5 seconds)              |
  |--- GET /orders/42/status ----------->|
  |<-- 200 "picked up" (news!) ----------|

In code it is a loop around fetch. The notes use setInterval(); a chain of setTimeout() calls is safer, because the next poll starts only after the previous one has finished. With setInterval, a slow server makes requests pile up and overlap.

let etag = null;
async function poll() {
  try {
    const res = await fetch('/orders/42/status', {
      headers: etag ? { 'If-None-Match': etag } : {},
    });
    if (res.status === 200) {
      etag = res.headers.get('ETag');
      render(await res.json());
    } // 304 Not Modified: nothing changed, nothing to do
  } finally {
    const base = document.hidden ? 60_000 : 5_000; // slow down in background tabs
    setTimeout(poll, base + Math.random() * 1_000); // jitter: clients do not poll in lockstep
  }
}
poll();

Advantages. It is very simple to build and to reason about. It works with any browser, any HTTP server, any proxy, any CDN and any firewall, with no special protocol. Each request is independent and stateless, so any server can answer it and load balancing needs nothing special.

Disadvantages.

• High server overhead. Most answers are empty. The notes say up to 99% of responses can carry nothing, wasting server CPU and bandwidth.
• Latency. It is not truly real time. With a 5-second interval, news can wait up to 5 seconds.
• Header overhead. Every poll sends the full set of HTTP headers both ways: cookies, auth tokens, user agent. That is often 500–1,000 bytes per round trip, much more than the answer “no change”. HTTP/2 header compression helps, but each poll is still a full request that the server must authenticate and route.

#Short polling in numbers

Events happen at random moments, so on average an event waits half the interval before the next poll picks it up, and at worst a whole interval (plus one round trip). The fleet-wide cost is simple division:

requests per second = clients ÷ interval

The worked example from the notes’ warning:

1. 1,000,000 open order pages, polling every 5 s: 1,000,000 ÷ 5 = 200,000 requests per second, whether or not anything happens.
2. Each order changes about once every 10 minutes (600 s). Only about 1 poll in 600 ÷ 5 = 120 finds news, so about 99.2% of answers are empty.
3. At about 800 bytes of headers per poll: 200,000 × 800 B = 160 MB/s, about 1.3 Gbps of traffic that says “no change”.
4. Average delay: 2.5 s; worst: about 5 s.

Halving the interval halves the delay and doubles every one of these costs. That trade-off is the whole story of short polling.

#Making short polling cheaper

• Conditional requests. Send the version you have in If-None-Match (or a timestamp in If-Modified-Since). If nothing changed, the server answers 304 Not Modified with no body (lesson 2.4). This saves bandwidth and serialisation, but not the request itself, the TLS work or the auth check, and the version check must be cheap (a counter in memory or Redis, not a database query).
• Cache shared answers at a CDN. If many users poll the same resource, such as a match score, give it Cache-Control: max-age=2. The CDN answers almost all polls; your origin sees about one request per two seconds per edge location, however many fans are watching. Short polling plus a CDN is one of the cheapest ways to update millions of screens with identical data.
• Adaptive intervals. Poll fast while something is happening (the order is out for delivery), slowly when nothing is (being cooked), and stop or slow right down when the tab is hidden (document.hidden). A server can say “ask me again in 30 s” with a Retry-After header or a field in the response.
• Jitter. If every client polls on the same schedule (after a deploy, every page reloads at once), the load arrives in synchronised spikes. A random offset spreads it out.

#When short polling is the right choice

When updates are rare, a delay of several seconds is fine, and the number of clients is modest: “is my export ready?”, a payment-status page, a status widget on an admin dashboard, a CI build badge. And when the data is shared by everyone and can sit behind a CDN. In these cases its simplicity beats everything else. Many production systems that “need real time” are well served by a 10-second poll.

#Long polling

Long polling (also called the hanging GET, and part of what was once called Comet) was introduced to cut the waste of short polling while keeping plain HTTP. It gives a near real-time experience with the ordinary request-response model.

Client                                  Server
  |--- GET /updates?after=41 ----------->|  (server holds the request open...)
  |                                      |  (event 42 happens!)
  |<-- 200 [42] -------------------------|  (response sent, request complete)
  |--- GET /updates?after=42 ----------->|  (asks again at once; server holds again...)

How it works, step by step:

1. The client sends an HTTP request.
2. The server does not answer at once. It keeps the request open (held) and waits.
3. As soon as there is new data, or a timeout passes, the server sends the response and that request is complete.
4. The client immediately sends a new long-poll request, and the cycle repeats.

Advantages. Far fewer requests and much less wasted server work than short polling. News is delivered as soon as it exists, so latency is low (about one network trip). It still uses ordinary HTTP, so it passes through almost every proxy and firewall.

Disadvantages.

• The request/response cycle repeats. The notes describe this as the friction of constantly opening and closing HTTP connections. In practice the client reuses one keep-alive TCP connection for poll after poll, so it is not a new TCP or TLS handshake each time. The real costs are a full request and response (with headers) per event or timeout, and a request held open on the server the whole time.
• Message ordering and gaps. When events come quickly, or the client retries, answers can arrive out of order, be duplicated, or miss events that happened between two polls. This needs a cursor (below).

Long polling is still common where the client is a program rather than a browser: AWS SQS ReceiveMessage with WaitTimeSeconds up to 20 s, the Telegram Bot API’s getUpdates with a timeout, and service-discovery “blocking queries”. Early web chat (Facebook Chat in 2008, for example) ran on it. Today it is mostly the fallback inside libraries that prefer a WebSocket or SSE but must work on any network (Socket.IO starts with long polling and upgrades; SignalR tries WebSockets, then SSE, then long polling).

#Timeouts: hold for less than the shortest timer on the path

A request cannot be held forever. Every box on the path closes connections that are quiet for too long:

If the server holds a poll longer than the shortest of these, the client gets a 504 Gateway Timeout or a reset instead of an answer. So servers hold for 25–30 s, then answer with an empty 204 No Content (or 200 with an empty list), and the client asks again. The client’s own timeout must be longer than the server’s hold, or it will abandon polls that were about to succeed and start a second one in parallel.

Requests per client are now events + timeouts. An idle client costs about two requests a minute instead of twelve with 5-second short polling, and a client that receives an event every few seconds sends roughly one request per event.

#The gap between polls, and the cursor that fixes it

After the server answers, there is a moment (a few milliseconds, or seconds on a slow phone) before the next request arrives. An event that happens in that gap has no request to ride on. If the server only forwards “live” events to waiting requests, that event is silently lost.

The fix is a cursor:

• The server gives every event an increasing ID and keeps recent events in a short log (a ring buffer in memory, a Redis stream, a database table).
• The client sends the last ID it has processed: GET /updates?after=42.
• If the log already has something newer, the server answers at once with everything after 42, in order, as one batch. Otherwise it holds the request.
• The client applies events in ID order and ignores IDs it has already seen, which also fixes duplicates and out-of-order answers when two polls cross (for example a retry fired while the old poll was still open). Keep one poll in flight per client.

How long to keep the log is a product decision: long enough to cover a typical disconnect (minutes), with a “too old, reload everything” answer for clients that come back after longer.

#What a held request costs the server

Each waiting poll is an open connection plus a little state. On an event-loop server (Nginx, Node.js, Go, Java with virtual threads, Python asyncio), that is a few kilobytes of memory, so one machine can hold tens or hundreds of thousands. On a thread-per-request server (Apache prefork, a classic servlet pool of 200 threads), every waiting poll occupies a whole thread, and the 201st client waits for a thread before it can even start waiting. That is the C10K problem from lesson 1.13: long polling needs an asynchronous server.

Two more design points:

• The event and the waiting request are often on different servers. The order service on server A learns the rider picked up; the client’s poll is waiting on server B. Servers share events through a pub/sub channel (Redis pub/sub, a message broker) so whichever server holds the request can answer it. Lesson 2.6 develops this for WebSockets; it is the same pattern.
• Retry with back-off and jitter. On a network error or a 5xx, wait a random, growing delay before polling again. A client that retries in a tight loop turns a server hiccup into a flood.

#Server-sent events (SSE)

Server-sent events are a one-way (unidirectional) channel: only the server sends, continuously, to the client. They are part of the HTML standard and run on plain HTTP: the client sends one ordinary GET, and the server answers with a response that simply never ends. Whenever there is news, the server writes another small block of text into that response.

Client                                       Server
  |--- GET /orders/42/stream ------------------>|
  |    Accept: text/event-stream                |
  |<-- 200 OK ----------------------------------|
  |    Content-Type: text/event-stream          |
  |    Cache-Control: no-cache                  |
  |<=== event 101 ==============================|   (the response stays open)
  |<=== event 102 ==============================|
  |<=== event 103 ==============================|

How it works:

• The client uses the browser’s EventSource API, which sends a normal HTTP GET.
• The request carries Accept: text/event-stream. This is only a hint (EventSource sends it by itself); what makes it a stream is that the server answers with Content-Type: text/event-stream.
• The server keeps the response open and streams each event as it happens. The notes mention Connection: keep-alive here; in HTTP/1.1 persistent connections are already the default, and HTTP/2 and HTTP/3 forbid that header altogether, so it is not what makes SSE work. What matters is that the server sends the headers at once and never ends the body.

#The wire format

The body is UTF-8 text made of events separated by a blank line. Each line is field: value:

retry: 5000

id: 101
event: status
data: {"order":42,"status":"cooking"}

: this line is a comment, used as a heartbeat

id: 102
event: status
data: {"order":42,"status":"picked_up"}

data: first line of a plain message
data: second line of the same message

The format is deliberately trivial: you can produce it with print from any language, read it in a terminal with curl, and debug it in the browser’s network panel (Chrome shows each event in an EventStream tab).

#The client: EventSource

const es = new EventSource('/orders/42/stream'); // { withCredentials: true } to send cookies cross-origin

es.addEventListener('status', (e) => render(JSON.parse(e.data), e.lastEventId));
es.onmessage = (e) => console.log('event without a type:', e.data);
es.onerror = () => {
  // readyState 0 = reconnecting by itself; 2 = closed for good (it got a non-200 answer)
  if (es.readyState === EventSource.CLOSED) setTimeout(reopen, 1_000 + Math.random() * 10_000);
};
// es.close() when the page no longer needs updates

#The server

Any server that can write to a response without closing it can serve SSE. A complete one in Node.js, with IDs, replay and heartbeats:

import http from 'node:http';

const log = []; // recent events, for replay after a reconnect
const clients = new Set();
let nextId = 1;
const frame = (ev) => `id: ${ev.id}\nevent: status\ndata: ${ev.data}\n\n`;

function publish(data) {
  const ev = { id: nextId++, data: JSON.stringify(data) };
  log.push(ev);
  if (log.length > 1000) log.shift();
  for (const res of clients) res.write(frame(ev));
}

http
  .createServer((req, res) => {
    if (req.url !== '/stream') return res.writeHead(404).end();
    res.writeHead(200, {
      'Content-Type': 'text/event-stream',
      'Cache-Control': 'no-cache',
      'X-Accel-Buffering': 'no', // tell Nginx not to buffer this response
    });
    res.write('retry: 5000\n\n');
    const last = Number(req.headers['last-event-id'] ?? 0);
    for (const ev of log) if (ev.id > last) res.write(frame(ev)); // replay what was missed
    clients.add(res);
    const beat = setInterval(() => res.write(': ping\n\n'), 20_000);
    req.on('close', () => {
      clearInterval(beat);
      clients.delete(res);
    });
  })
  .listen(8080);

setInterval(() => publish({ status: 'tick', at: Date.now() }), 3_000);

#Features: automatic reconnection and event IDs

These are what SSE gives you for free compared with a hand-made stream:

• Automatic reconnection. If the connection breaks because of a network error, the browser reconnects by itself after the retry delay (a few seconds by default, chosen by the browser).
• Built-in event IDs. On reconnect the browser sends Last-Event-ID with the last ID it received, so the server can send everything it missed. The browser only remembers the ID; the server must keep the history and decide how far back it can replay. A server that never sends id: lines gets no resume at all; the alternative is to send a full snapshot (the order’s current state) as the first event of every connection.
• Text data. SSE is meant for UTF-8 text and JSON.

Two rules are easy to miss. First, if the response is not a 200 with text/event-stream (a 502 or 503 during a deploy, a 401, an HTML error page), EventSource fails the connection and never retries; your onerror handler must create a new one after a random delay. Second, the server can end a stream on purpose by answering 204 No Content, which tells the browser to stop.

#Advantages and limits

Advantages. SSE runs on HTTP/1.1, HTTP/2 or HTTP/3, so there is no protocol switch and usually no firewall change: the same load balancers, auth, logging and CDNs (if they allow streaming) keep working. It is easy to build, needs one request for the whole life of the page, has reconnection and resume built in, and the per-event overhead is a few bytes instead of a set of headers.

Limits.

• One direction only. The client cannot send data on the stream. To send something it makes a separate HTTP request (a POST) as usual. For most features (notifications, feeds, progress, AI answers) that is fine; for chat with typing indicators or a multiplayer game, a WebSocket is the better fit (lesson 2.6).
• UTF-8 text only. Binary data must be encoded (base64 adds about 33%) or, better, sent as a URL the client fetches separately.
• Six connections per origin on HTTP/1.1. Browsers open at most six HTTP/1.1 connections to one origin, shared by all tabs. Each SSE stream holds one for as long as the page is open, so the seventh tab’s stream, and every other request from that tab to the same origin, waits forever. HTTP/2 and HTTP/3 remove the problem by multiplexing many streams over one connection (servers usually allow 100 or more at once). Browser makers consider the limit intentional and will not raise it, so serve SSE over HTTP/2, or have one tab (or a SharedWorker) hold the stream and share events with the others through a BroadcastChannel.

#SSE in production

Most SSE bugs are not in your code but in the boxes between your code and the browser, which were built for responses that end.

#Buffering in proxies and compression

A reverse proxy normally buffers responses: Nginx reads the app’s output into memory buffers and sends it on in large pieces (lesson 1.13). For a stream of tiny events, that means nothing reaches the browser until a buffer fills or the stream ends, and then everything arrives at once. The classic symptom: it works on localhost and arrives in bursts in production.

• Send X-Accel-Buffering: no from the app (Nginx honours it per response), or set proxy_buffering off for the stream’s location.
• Check every other hop: some CDNs, API gateways and serverless platforms only return complete responses. Look for “response streaming” support before you build on one.
• Compression buffers too. A gzip or Brotli middleware waits for a full block before emitting anything. Exclude text/event-stream from compression, or flush the compressor after each event. Events are small, so you lose little.
• Corporate antivirus proxies sometimes buffer whole responses to scan them. You cannot fix their configuration, which is one reason libraries keep long polling as a fallback.

location /stream {
    proxy_pass http://app;
    proxy_http_version 1.1;
    proxy_set_header Connection '';   # keep the upstream connection persistent
    proxy_buffering off;              # or let the app send X-Accel-Buffering: no
    proxy_cache off;
    gzip off;
    proxy_read_timeout 1h;            # heartbeats keep it busy anyway
}

#Idle timeouts and heartbeats

A stream with no events for a minute looks dead to the load balancer, the proxy and NAT devices, and they close it (60 s is a common default). The browser reconnects, so it mostly works, but every client reconnects every minute and anything in between depends on replay. The fix is a heartbeat: write a comment line (: ping) every 15–30 s, safely below every idle timeout on the path. As a bonus, a heartbeat write that fails tells the server the client is gone, so it can free the connection instead of keeping a ghost subscriber.

#Authentication: EventSource cannot set headers

EventSource takes only a URL and one option, withCredentials. It cannot send a body, a method other than GET, or custom headers such as Authorization: Bearer …. The workable choices:

#Server resources, scaling and deploys

The notes say SSE has low backend overhead. Per event that is true: a few bytes, no headers, no request parsing. But every connected client is an open connection held for minutes or hours:

• Use an event-loop server (as for long polling). Raise the process’s file-descriptor limit; each connection is a socket.
• Plan for the count: 1,000,000 connected users are 1,000,000 open connections, spread over enough servers and load-balancer capacity. Heartbeats every 20 s for a million clients are 50,000 small writes a second, cheap but not free.
• Fan-out: an event produced on one server must reach clients connected to all the others. Publish it to a pub/sub channel that every SSE server subscribes to (lesson 2.6 and Chapter 30–32).
• Deploys: a restart drops every stream on that server. Drain gracefully: stop accepting new streams, close existing ones in batches with a randomised retry:, and keep the event log outside the process so any server can replay Last-Event-ID.

#SSE for LLM token streaming

When you ask ChatGPT, Claude or Gemini a question, the answer appears word by word. A long answer can take 10–30 seconds to generate, but the first words appear within about a second. That stream is, in almost every case, SSE: the OpenAI and Anthropic APIs, among others, return text/event-stream when the request sets "stream": true.

Why SSE and not a WebSocket?

• The data flows one way. The prompt goes up once in a normal POST; tokens flow down. Nothing needs to travel upstream mid-answer except “stop”, and closing the connection says that.
• One request, one answer. It maps exactly onto HTTP: the same API keys, auth, rate limits, load balancers, logging and billing as every other API call. A WebSocket would need its own connection management and message routing.
• Time to first token is what users feel. Streaming turns a 15-second wait into a 0.5-second wait followed by reading.

What the stream looks like differs by provider. The Anthropic Messages API sends typed events (message_start, content_block_delta with a text chunk, message_stop, plus occasional ping events); OpenAI’s Chat Completions API sends data: {…} chunks and ends with data: [DONE]. Because these requests are POSTs with an Authorization header, clients read them with fetch streaming, not EventSource.

A typical product keeps the API key on its own backend, which reads the model’s stream and forwards each chunk to the browser as its own SSE stream. Three things go wrong there:

1. Buffering anywhere on the path (proxy, compression, a gateway) turns the stream into one late block: the most common bug in AI features.
2. Cancel must propagate. When the user presses stop, the browser aborts its fetch. Your backend must notice the disconnect and close the upstream request too, or it keeps paying for tokens nobody reads.
3. There is no resume from the model. If the connection drops halfway, the model APIs cannot continue from an event ID. Either retry the whole request (costly, and the answer may differ) or let the backend keep generating into a store keyed by a response ID, so the page can re-attach with Last-Event-ID to your stream.

#When the client is not a browser tab: webhooks and mobile push

Polling and SSE need the client to be running and connected. Two other push channels cover the cases where it is not.

#Webhooks: server-to-server push

A webhook is an HTTP POST that one company’s server sends to a URL you registered, when something happens on their side: a payment succeeded (Stripe, bKash and other gateways), a commit was pushed (GitHub), a message was delivered. Your “client” here is your own server, which is publicly reachable, so the provider can simply call it. Instead of polling the provider’s API every minute, you are told.

Webhooks are at-least-once: the provider retries until you answer 2xx, so you will see failures, delays and duplicates. A solid receiver:

• Verifies the signature. The provider signs each request with a secret shared only with you, usually an HMAC-SHA256 of the timestamp and the raw body (for example Stripe’s Stripe-Signature: t=…,v1=… or GitHub’s X-Hub-Signature-256). Recompute it over the raw bytes (not re-serialised JSON), compare in constant time, and reject timestamps older than a few minutes (Stripe’s libraries default to 5) so a captured request cannot be replayed later. Your URL is public; without this anyone can POST “payment succeeded”.
• Is idempotent. Store each event ID with a unique key and skip IDs you have already processed. Answer 200 for duplicates too, or the provider keeps retrying.
• Answers fast and works later. Put the event on a queue and return 2xx within a second or two. Providers time out after a few seconds (GitHub waits 10) and treat a slow answer as a failure, which causes the duplicates you just learned to handle.
• Does not trust arrival order. After retries, “refunded” can arrive before “paid”. Use the event’s timestamp or fetch the current state from the provider’s API.
• Reconciles. Retry policies differ: Stripe retries for up to three days with exponential back-off; GitHub does not retry automatically (you redeliver by hand). Run a periodic job that lists recent events from the provider’s API to catch anything that never arrived.

#Mobile push: APNs and FCM

A phone in a pocket cannot keep an SSE stream or a long poll open for every app: the operating system suspends background apps and closes their connections to save battery. Instead, the OS keeps one connection to its platform’s push service, and every app’s notifications travel over it:

• APNs (Apple Push Notification service) for iPhone, iPad and Mac; FCM (Firebase Cloud Messaging) for Android, and also usable for iOS and the web. Browsers have the equivalent Web Push, which wakes a service worker.
• The app registers and receives a device token, which it sends to your server. To notify, your server calls APNs or FCM with the token and a small payload (about 4 KB at most).
• Delivery is best effort: a push can be delayed, collapsed into the latest one with the same collapse key while the phone is offline, or dropped. A token goes stale when the app is uninstalled (APNs answers 410 Unregistered); delete it.
• So treat a push as a wake-up call, not as the data: the app wakes and syncs from your API with its cursor, exactly like a long poll after a reconnect.

#Reconnect storms

Long polls and SSE streams are long-lived, so the dangerous moment is not normal traffic but everyone reconnecting at once: after a deploy, a server crash, a load-balancer failover or a network blip. A reconnect is far more expensive than an idle connection: TCP and TLS handshakes, an auth check, a replay query from Last-Event-ID. If 50,000 clients do it in the same second with a fixed retry delay, the remaining servers overload, fail health checks and drop their clients too, which then retry together again. The outage feeds itself.

The defence is the same everywhere (and comes back for retries in general in Chapter 28):

• Exponential back-off with full jitter on the client: after each failure, double the limit (1, 2, 4, 8… s, capped) and wait a random time between 0 and that limit. 50,000 reconnects spread over 30 s are about 1,700 a second.
• The server helps: send retry: with a randomised value, drain connections gradually before a deploy, and let the load balancer bring a restarted server up slowly.
• Shed load explicitly: a server that is overloaded should refuse quickly (a 503 with Retry-After) rather than accept connections it cannot serve. Remember that EventSource stops on a 503, so your onerror must restart it with back-off.
• Long polling has its own herd: a broadcast event answers every held poll at once, and all those clients re-poll in the same instant. A small random delay before re-polling after a broadcast fixes it; SSE does not have this problem because its connection simply stays open.

#Choosing between them, with numbers

The same fleet in numbers: 1,000,000 connected clients, one piece of news per client every 10 minutes:

A practical decision path:

1. Does the client need to send often too (chat with typing, games, collaborative editing)? Use a WebSocket (lesson 2.6).
2. Is the receiver another server? Use webhooks. A phone with the app closed? Mobile push, then sync.
3. Is a delay of several seconds fine, or is the data the same for everyone? Short polling, with conditional requests and a CDN where possible.
4. Otherwise (one-way updates that should feel instant): SSE, served over HTTP/2, with IDs, a replay log and heartbeats. Keep long polling as a fallback if your users sit behind proxies that break streaming.

#Real-time push in system design

What interviewers and production incidents ask about:

• Name the latency budget first. “Within 1 s” and “within 30 s” lead to different designs. Many requirements that say “real time” mean “a few seconds”.
• Separate the notification from the data. The push (a poll result, an SSE event, a mobile push, a webhook) says something changed; the client then reads the source of truth with a cursor. That makes duplicates, gaps and reconnects harmless.
• Every event needs an ordered ID and a short replay log. Without them you cannot resume after a disconnect, deduplicate, or order events.
• Stateful connections change scaling. Long polls and SSE streams pin a client to a server for minutes. You need a pub/sub fan-out between servers, connection-aware load balancing, file-descriptor and memory budgets, and graceful draining for deploys.
• Count the cost: requests per second for polling, open connections for long polling and SSE, and reconnects per second after a failure.
• Plan for the storm: back-off with jitter, randomised retry:, gradual drains, load shedding.
• Know your boxes: proxy buffering, compression, idle timeouts and the six-connection limit break SSE far more often than application code.

#Try it yourself

# 1. Watch a real public SSE stream: Wikimedia's live feed of edits. -N turns off curl's buffering.
curl -N -A 'sse-lesson/1.0 (you@example.com)' \
  https://stream.wikimedia.org/v2/stream/recentchange | head -n 20
#   event: message
#   id: [{"topic":"…","partition":0,"timestamp":…}]
#   data: {"$schema":"/mediawiki/recentchange/1.0.0","meta":{…},"type":"edit",…}

# 2. Run the Node.js server from this lesson (node server.mjs), then:
curl -N http://localhost:8080/stream                        # live events every 3 s, heartbeats every 20 s
curl -N -H 'Last-Event-ID: 5' http://localhost:8080/stream  # replays everything after id 5 first

# 3. Conditional short polling: the second request should get 304 if nothing changed
curl -si https://example.com/ | grep -i etag
curl -si -H 'If-None-Match: "<etag from above>"' https://example.com/ | head -n 1

# 4. A long poll you can watch being held: SQS waits up to 20 s for a message
aws sqs receive-message --wait-time-seconds 20 \
  --queue-url https://sqs.us-east-1.amazonaws.com/123456789012/orders

In the browser, open the developer tools console on any page and run:

const es = new EventSource('https://stream.wikimedia.org/v2/stream/recentchange');
es.onmessage = (e) => console.log(JSON.parse(e.data).title);
// later: es.close()

Then open the Network panel, click the recentchange request and look at its EventStream tab. Turn the network off in the panel’s throttling menu for a few seconds and back on: watch the request fail, a new one start by itself, and its request headers carry Last-Event-ID.

#Summary

Key takeaways

• HTTP is client-initiated and browsers are unreachable behind NAT, so every push technique arranges for a request or connection from the client to be waiting when news appears. The five main ones are short polling, long polling, SSE, WebSockets and WebRTC; outside the tab there are webhooks and mobile push.
• Short polling asks every interval: delay is interval ÷ 2 on average, up to a whole interval; cost is clients ÷ interval requests per second, mostly empty (1M clients every 5 s = 200,000 req/s, about 99% empty). Make it cheaper with If-None-Match/304, CDN caching of shared data, adaptive intervals and jitter. Chain setTimeout rather than setInterval.
• Long polling (hanging GET) holds the request until news or a timeout, then the client asks again at once over the same keep-alive connection. Hold 25–30 s, below every proxy and load-balancer timeout (often 60 s). Use a cursor (after=<last ID>) and a server-side log so events in the gap are not lost; keep one poll in flight; retry with jittered back-off; use an event-loop server.
• SSE is one never-ending text/event-stream response with data, event, id, retry and comment lines. EventSource reconnects by itself and sends Last-Event-ID; the server must keep the log. It is one-way, UTF-8 only, and limited to six connections per origin on HTTP/1.1 (use HTTP/2). A non-200 answer stops EventSource for good.
• In production SSE breaks on proxy buffering (X-Accel-Buffering: no), compression, idle timeouts (heartbeat every 15–30 s) and auth (EventSource cannot set headers: cookies, a short-lived ticket, or fetch streaming).
• LLM APIs stream tokens over SSE because the data is one-way, one request maps to one answer, and time to first token is what users feel. Propagate cancel upstream; there is no resume from the model.
• Webhooks are at-least-once server-to-server POSTs: verify the HMAC signature over the raw body with a timestamp, deduplicate on the event ID, answer fast and queue the work. APNs/FCM carry best-effort mobile pushes over the OS’s single connection: treat a push as a wake-up call and sync with a cursor.
• After restarts, prevent reconnect storms with exponential back-off and full jitter, randomised retry: and gradual draining.

Self-check: polling, long polling and SSE30 questions