The Event Loop and Virtual Threads

#Where this lesson fits

Lesson 1.2 compared the ways a server handles many clients and lesson 1.13 showed Nginx’s master and workers. Lesson 2.5 and lesson 2.6 showed why long-lived connections need something cheaper than a thread each. In this chapter, lesson 3.1 defined threads, lesson 3.2 showed what a context switch costs, lesson 3.3 covered thread pools and lesson 3.5 showed how epoll tells a program which sockets are ready.

This lesson builds on all of that. epoll only answers “what is ready?”. The event loop is the program structure that sits on top of it: what runs, in what order, and what happens when something takes too long. Then we look at the other answer to the same problem, virtual threads, and at how to choose. Races and locks in code belong to Chapter 14.

#What an event loop is

An event loop is a programming model in which one loop keeps taking events and running the callback or task for each one. JavaScript and Node.js, Python’s asyncio and many other event-driven runtimes are built on it. The notes draw it like this:

         Event created
               |
               v
          Event queue
               |
               v
           Event loop
               |
               v
        Execute callback
               |
               v
        Wait for events
               |
               +---- repeat

In code, the idea is only this:

while there is still work pending:
    events = ask the kernel what is ready, sleeping if nothing is
    for each event:
        run its callback to completion

Two rules follow, and the rest of this lesson is really about them:

1. Waiting costs almost nothing. A connection waiting for data is a small record in the kernel and a small object in your program, not a thread. So one thread can hold tens of thousands of mostly idle connections.
2. Working blocks everyone. Each callback runs to completion and nothing can interrupt it. While one runs, every other connection on that loop waits.

So an event loop gives you concurrency, not parallelism: many things in progress, one running at a time (lesson 3.2). It fits I/O-bound work, where a request spends most of its life waiting. It is a poor fit for CPU-bound work, where there is nothing to wait for and the work must simply be done.

#The smallest Node.js server

The notes’ example is a complete server:

const http = require("http");

const server = http.createServer((req, res) => {
  res.end("Hello World");
});

server.listen(3000);

There is no loop in this code, and no thread is created. server.listen registers a socket and the script ends, but the process does not exit, because the runtime’s event loop now has something to wait for. Your arrow function runs later, once per request.

#“Single-threaded” is only half true

Your JavaScript normally runs on one main thread. That does not make Node.js a single-threaded system. A running Node process has several threads, and three different parties do the work:

libuv is the C library underneath Node.js that provides the event loop and the worker pool. The pool has 4 threads by default, set with the UV_THREADPOOL_SIZE environment variable when the process starts. Why does it exist? Regular files have no useful readiness signal: a regular file always reports “ready” (lesson 3.5), and the read may still block on the disk. So libuv runs a plain blocking call on a pool thread and reports back when it finishes. The same trick covers getaddrinfo, which has no non-blocking version.

The most useful fact in the table: the pool is not used for network sockets. Ten thousand HTTP connections use no pool threads at all. But four slow file reads, or four stuck name lookups, fill the pool completely.

#Node’s event loop, phase by phase

The notes list six phases, and each one has its own queue of callbacks:

Three details are worth remembering:

• How long poll sleeps. If a setImmediate is waiting, poll does not sleep at all. Otherwise it sleeps until the next timer is due, or indefinitely if there are no timers. That is why an idle Node process uses almost no CPU.
• A timer’s delay is a minimum. setTimeout(fn, 100) means “not before 100 ms”. If a callback is running at that moment, the timer waits for it.
• When the process ends. After each turn the loop checks for live handles (a listening server, an open socket, a timer) and outstanding requests. If there are none, the process exits by itself. timer.unref() tells the loop not to count a timer.

The diagrams usually show timers first. Since libuv 1.45 (Node.js 20), timers run only after poll rather than both before and after it, as the Node.js documentation notes. The cycle is the same, so all the ordering rules below still hold, but the first turn differs slightly from older descriptions.

#Microtasks, and who runs next

Outside the phases, Node has two more queues, and these are what make ordering puzzles interesting:

• The process.nextTick queue, which belongs to Node.
• The Promise microtask queue, which belongs to the JavaScript engine: .then callbacks, code after an await, and queueMicrotask.

The notes mention both together with “a separate scheduling behaviour”. The precise rule is this:

1. Run the current piece of synchronous code to completion.
2. Empty the nextTick queue completely.
3. Empty the Promise microtask queue completely. If a microtask added more ticks, go back to step 2.
4. Only now run the next callback, whether in this phase or the next one.

Since Node.js 11 this check happens after every single callback, not only between phases. That matches how browsers behave.

#The notes’ example

console.log("Start");

setTimeout(() => {
  console.log("Timer");
}, 0);

Promise.resolve().then(() => {
  console.log("Promise");
});

console.log("End");

Start
End
Promise
Timer

Synchronous code first, so Start and End. Then the Promise, as a microtask. The timer last, because it needs the loop to reach a phase. A delay of 0 does not mean “now”: Node turns it into 1 ms, and either way it must wait for the stack to clear.

console.log("Start");
setTimeout(() => {
  console.log("Timer");
}, 0);
Promise.resolve().then(() => {
  console.log("Promise");
});
console.log("End");

#process.nextTick runs before Promises, usually

Add process.nextTick(() => console.log("Tick")) to that script and, in a classic script run by node order.js, it prints between End and Promise, even if the Promise was scheduled first. So the two are not one queue, which corrects the notes.

There is a trap. Rename the file to order.mjs and Promise prints before Tick. A JavaScript module’s top-level code itself runs from inside the microtask queue, so that queue finishes draining before Node looks at its tick queue. Both results are correct. Code that depends on this order is fragile, so do not write it. Prefer queueMicrotask for “soon”, as the Node.js documentation also recommends.

#setImmediate vs setTimeout(0)

Use setImmediate when you want “after the current batch of I/O” and when you break up long work, because it lets I/O run in between. Recursive nextTick or Promise chains starve the loop: their queues must be empty before anything else runs, so a queue that refills itself holds up timers and I/O for as long as it continues.

#await is not a thread

await does not block anything and does not start a thread. It saves the function’s state, ends the current task and schedules the rest for later. Everything between two awaits runs without interruption. Two consequences catch people out:

• An async function full of CPU work blocks the loop as much as any other function. async does not move work anywhere.
• With no threads there are no data races inside one callback, but logic races across an await are still possible: const x = await read(); await write(x + 1); can lose an update if another request interleaves between the two lines. Chapter 14 covers this.

#What blocks the loop

The notes’ example is the extreme case:

while (true) {
  // infinite loop
}

Other requests are not handled, timers do not fire and callbacks stay stuck. This is called blocking the event loop. Real incidents are rarely while (true). They look like this:

#The arithmetic of one thread

An event loop has a very simple capacity rule. If each request needs C milliseconds of time on the main thread, one loop can do at most 1000 / C requests per second:

Time waiting on the database does not count in C. That is the entire benefit. And near that limit latency explodes rather than rising gently, because arriving requests queue behind each other. Keep a loop well under its limit, about 60–70% busy at peak, so bursts have room.

A blocked callback also adds to everyone’s latency: if 1,000 requests per second are arriving and one callback holds the thread for 400 ms, 400 requests pile up and have to be drained afterwards.

#How to fix it

Worker threads and processes are different tools. cluster adds capacity by running one loop per core. Worker threads protect the loop from a heavy job. A healthy Node service often uses both.

#Event-loop lag: the metric to watch

CPU does not tell you if a loop is healthy. One blocked thread on a 16-core machine is about 6% total CPU. The direct measure is event-loop lag, also called event-loop delay: how late a timer fires compared with when it was due. If a timer is meant to fire every 20 ms and it fires after 320 ms, the loop was busy for about 300 ms, and any request arriving then waited as long.

Node has this built in:

const { monitorEventLoopDelay, performance } = require("node:perf_hooks");

const delay = monitorEventLoopDelay({ resolution: 20 });
delay.enable();
let last = performance.eventLoopUtilization();

setInterval(() => {
  const now = performance.eventLoopUtilization();
  const busy = performance.eventLoopUtilization(now, last).utilization;
  last = now;
  console.log(
    `lag p99 ${(delay.percentile(99) / 1e6).toFixed(1)} ms, ` +
      `busy ${(busy * 100).toFixed(0)}%`,
  );
  delay.reset();
}, 1000);

Read the two numbers together:

• Lag (nanoseconds in the histogram) shows how long a callback waited. An idle process reads about the timer resolution, not zero, so watch for growth above that baseline. Treat a sustained p99 of tens of milliseconds above it as a warning and hundreds as an incident; the right threshold depends on your latency target.
• Event loop utilization is the fraction of time the loop was doing work rather than sleeping in poll. It predicts saturation better than CPU does, because CPU also includes GC and other threads.
• Low utilization with occasional large lag spikes points to one bad callback. High utilization and rising lag point to too much load, so add processes.

Then find the culprit with a CPU profile (node --cpu-prof, or a flame graph from your APM). In production, export lag as a metric, alert on it, and consider shedding load when lag is high: returning 503 quickly is better than making everyone wait.

#Blocking, non-blocking and asynchronous

These three words are not synonyms. The notes’ table, sharpened a little:

Non-blocking and asynchronous are not the same thing. A non-blocking call returns immediately, whether or not it has done any work. An asynchronous operation reports its completion later. Node’s fs.readFile is asynchronous to your code, yet it is implemented with a blocking call on a pool thread. Sockets in Node are non-blocking underneath and appear to you as asynchronous. So “asynchronous” describes the interface your code sees, and “non-blocking” describes one system call. Lesson 3.5 goes through the kernel side.

#What event loops are good and bad at

Good:

• Very many concurrent connections that are mostly idle: chat, WebSockets, API gateways, proxies, streaming.
• Low memory use per connection, a few kilobytes, so high connection counts are affordable.
• Better use of resources, because while one request waits for I/O the thread serves others.
• One thread touching your data means no locks around in-memory state within a process.

Bad:

• CPU-heavy work, as shown above.
• Everything must cooperate. One library that blocks hurts all requests on that process, so isolation is poor.
• A single uncaught error can take down the whole process and all its connections.
• Callback-style code splits one logical operation across functions. async/await hides most of this, but stack traces and context propagation are still harder than in blocking code.

#asyncio: the same idea in Python

Python’s asyncio works the same way: one thread, one loop, coroutines that give the thread back at each await. On Linux it uses epoll underneath, and uvloop swaps in libuv itself for extra speed. The rule that catches most people:

import asyncio
import time


async def bad(name):
    time.sleep(1)  # blocks the thread, and the loop with it
    print(name)


async def good(name):
    await asyncio.sleep(1)  # gives the thread back while waiting
    print(name)


async def main():
    start = time.perf_counter()
    await asyncio.gather(good("a"), good("b"), good("c"))
    print(f"{time.perf_counter() - start:.1f} s")  # about 1.0, with bad() about 3.0


asyncio.run(main())

So a blocking call hidden inside async def, such as requests.get or a synchronous database driver, stalls every coroutine. Use an async library or push the call to a helper thread with await asyncio.to_thread(func). Run with asyncio.run(main(), debug=True) while developing and asyncio logs any callback slower than 100 ms. For CPU work in Python, threads are limited further by the GIL (lesson 3.2), so the usual answer is a process pool.

#Other event loops: Nginx and Redis

The notes group Node.js, Nginx and Redis together as a “single-threaded architecture”. All three use event loops, but they are built differently, and the difference matters in a design.

#Nginx: one loop per worker process

Nginx is not one event loop. It is one event loop per worker process, normally one worker per core (worker_processes auto;). Lesson 1.13 showed the master and the configuration; three more things matter here:

• Each connection belongs to one worker for its whole life. There is no shared request queue, so workers rarely need locks and the design scales nearly linearly with cores.
• Nginx runs no application code, only quick parsing, routing and copying, so its callbacks are short by nature. A slow upstream only means a socket stays in the epoll set for longer.
• The weak spot is disk. Reading a file that is not in the page cache blocks the whole worker. aio threads; sends those reads to a thread pool. It is exactly libuv’s trick in another product.

#Redis: one thread executes commands

“Redis is single-threaded” is true of command execution and of nothing else:

Why choose one execution thread? Commands are in-memory and very fast, so the CPU is rarely the limit; there are no locks to take, and every command is atomic without any effort. Simple operations reach the order of 100,000 per second per instance on one core. To use more cores, you run more instances and partition the keys with Redis Cluster.

The event-loop rule applies exactly: a slow command freezes every client. KEYS *, DEL on a huge key, a large SMEMBERS, or a long Lua script all stop everything. So use SCAN, UNLINK and small batches, and watch SLOWLOG. Chapter 23 covers Redis as a cache.

#The other answer: cheap threads

Event loops solve the cost of waiting but make you pay in code: callbacks or async/await, and the rule that nothing may block. There is another route. Threads are expensive because of how the kernel implements them, so what if the runtime implemented them instead?

Why OS threads cost so much at this scale (lesson 3.1, lesson 3.5):

• Each has a large fixed stack reservation (typically 1 MB of address space for a Java thread on 64-bit Linux, of which the touched part stays resident).
• Creating one takes a system call and on the order of tens of microseconds.
• Switching between them goes through the kernel scheduler and disturbs CPU caches.

Green threads, or user-level threads, are scheduled by the language runtime rather than by the kernel. Many of them run on a few OS threads, which is called M:N scheduling. The idea is old: very early Java had green threads and dropped them, because they all ran on one OS thread and could not use several CPUs. What changed is that modern versions combine M:N scheduling with a non-blocking I/O layer built into the runtime.

Here is the key insight: underneath, these runtimes are event loops. When a goroutine or a virtual thread reads a socket, the runtime registers it with epoll and runs something else. The difference is who writes the suspend and resume. With async/await you mark every suspension point yourself. Here the runtime does it, so ordinary-looking code just works.

This also removes the “coloured functions” problem: in JavaScript or Python, async functions and plain functions are two kinds that do not mix freely, and one async call deep inside forces every caller up the stack to change. With goroutines and virtual threads, all functions are the same colour.

#Goroutines

In Go, every go f() starts a goroutine. They are the only concurrency model the language offers:

• Start small. A goroutine’s stack starts at about 2 KB and grows by copying when needed.
• M:N scheduling. The runtime runs goroutines on threads, with GOMAXPROCS threads running Go code at once. It defaults to the CPU count, and since Go 1.25 the default on Linux also respects a container’s CPU limit, which fixed a long-standing cause of throttling.
• Work stealing. Each logical processor has a local run queue, and an idle one takes work from a busy one.
• Network I/O goes through the runtime’s poller (epoll, kqueue, IOCP), so a waiting goroutine uses no thread.
• Blocking system calls do hold a thread, so the runtime gives the processor to another thread, creating one if needed.
• Preemption. Since Go 1.14 a long-running goroutine is interrupted with a signal after about 10 ms, so a tight loop cannot monopolise a thread. An event-loop callback gets no such protection.

Unlike one event loop, this uses all cores with one program. In return, goroutines truly run in parallel, so shared data needs locks or channels, and data races are real (Chapter 14).

go func() {
  resp, err := http.Get(url)
  if err != nil { return }
  defer resp.Body.Close()
  results <- resp.StatusCode
}()

#Java virtual threads

Virtual threads became final in Java 21 through JEP 444 (September 2023), after previews in Java 19 and 20. Java 25 is the current long-term support release at the time of writing, with Java 21 still widely deployed. Their design goal is that you keep existing blocking code, but it no longer ties up an OS thread while waiting:

import java.time.Duration;
import java.util.concurrent.Executors;
import java.util.stream.IntStream;

public class VT {
  public static void main(String[] args) {
    long start = System.currentTimeMillis();
    try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
      IntStream.range(0, 100_000).forEach(i ->
          executor.submit(() -> {
            Thread.sleep(Duration.ofSeconds(1));
            return i;
          }));
    }
    System.out.println("done in " + (System.currentTimeMillis() - start) + " ms");
  }
}

That runs 100,000 concurrent one-second sleeps and finishes in about 2 seconds on a laptop. With one OS thread each, most machines could not start them all.

How it works:

• A virtual thread mounts on a carrier, a normal OS thread from a small ForkJoinPool with one carrier per core by default.
• When it blocks in a JDK call that supports this (socket, sleep, lock, queue), the JVM copies its stack to the heap and it unmounts. The carrier picks up another virtual thread.
• When the wait ends, it becomes runnable and mounts again, perhaps on a different carrier.

Frameworks now expose this with a switch, such as spring.threads.virtual.enabled=true in Spring Boot 3.2 and later. Four rules follow from the design:

1. Do not pool virtual threads. Pools exist because threads are expensive; these are not. Create one per task.
2. Limit the scarce resource instead. A 200-thread pool was also, by accident, a limit of 200 database calls at once. With virtual threads you need an explicit Semaphore or a bounded connection pool, or a spike goes straight through to your database.
3. Watch out for pinning. See below.
4. Be careful with ThreadLocal caches. Code that assumed a few hundred threads and stores a large buffer in each can use huge amounts of memory with a million.

Pinning means a virtual thread cannot unmount, so its carrier blocks too:

Because there are few carriers, pinning is serious: with all of them pinned, every virtual thread stops even though the CPU is idle. On Java 21, find these with the JFR event jdk.VirtualThreadPinned and replace synchronized around blocking calls with ReentrantLock, or move to JDK 24+.

try (var ex = Executors
    .newVirtualThreadPerTaskExecutor()) {
  ex.submit(() -> {
    var body = httpGet(url); // waits
    save(body);
  });
}

Virtual threads make waiting cheap; they do not make computing faster. They have no time slicing among themselves today, so CPU-bound tasks should stay on a fixed pool sized to your cores (lesson 3.3).

#The wider family

#How to choose

A decision path that holds up in practice:

1. Mostly CPU? A thread pool (or processes) with about one thread per core. Virtual threads and event loops do not help.
2. Mostly waiting, with many connections? Any of the cheap options works. Pick by what your team and libraries already use, not by benchmarks, because at this level the database almost always limits you first.
3. Mixed? Keep the I/O on the loop or on virtual threads, and send CPU work to a bounded pool. Separate them into different services once the CPU work is big enough to scale on its own.
4. Only a few hundred concurrent requests? A plain thread pool is fine. Do not rewrite a working service to save threads you are not short of.

A real architecture often uses all three: Nginx (event loop) at the edge, a Node.js or Java virtual-thread API behind it, and a CPU-sized worker pool for image processing behind a queue.

#In an interview

• If someone says “Node can’t scale because it’s single-threaded”, separate concurrency from parallelism, name the limit (1000 / C per process), and say you would run one process per core behind a load balancer.
• For a WebSocket hub with a million idle connections, choose an event loop or goroutines, then point out the real limits: memory per connection, file descriptors, and fan-out work (lesson 2.6).
• If someone proposes virtual threads as a performance win, ask what the workload is. They improve throughput for waiting work, not latency for CPU work, and they remove the implicit limit that a thread pool gave you.
• Asked why Redis is “single-threaded but fast”, say: in memory, no locks, epoll, and short commands. Then add what breaks it: one slow command stops all clients.
• Always mention what you would measure: event-loop lag and utilization, pool queue depth, pinned-thread events, goroutine count.

#Try it yourself

Everything here needs only Node.js. The order and pool results below were checked on Node.js 24.

1. Predict, then run. Save as order.js, run node order.js, then copy it to order.mjs and run that. Only the position of Tick changes.

console.log("Start");
setTimeout(() => console.log("Timer"), 0);
Promise.resolve().then(() => console.log("Promise"));
process.nextTick(() => console.log("Tick"));
console.log("End");

2. See the pool limit. Save as pool.js. Four hashes finish at about the same time and the fifth takes about twice as long. Then try UV_THREADPOOL_SIZE=5 node pool.js (Linux or macOS) and the fifth catches up, if you have enough cores.

const crypto = require("node:crypto");
const start = Date.now();
for (let i = 1; i <= 5; i++) {
  crypto.pbkdf2("secret", "salt", 500_000, 64, "sha512", () => {
    console.log(`hash ${i} done at ${Date.now() - start} ms`);
  });
}

3. Block a server and watch lag. Save as lag.js and run it.

const http = require("node:http");
const { monitorEventLoopDelay } = require("node:perf_hooks");

const delay = monitorEventLoopDelay({ resolution: 20 });
delay.enable();
setInterval(() => {
  console.log(`lag max ${(delay.max / 1e6).toFixed(0)} ms`);
  delay.reset();
}, 1000);

http
  .createServer((req, res) => {
    if (req.url === "/block") {
      const end = Date.now() + 3000;
      while (Date.now() < end) {} // blocks everything
    }
    res.end("Hello World\n");
  })
  .listen(3000);

In a second terminal run the first command below, and immediately the second in a third terminal. The harmless / request takes almost 3 seconds, and lag jumps to about 3,000 ms.

curl http://localhost:3000/block
curl -w "took %{time_total}s\n" http://localhost:3000/

4. Starve the loop. Run this, then change the marked line and compare. The timer does not run until the nextTick chain ends; with setImmediate it runs almost immediately.

setTimeout(() => console.log("timer ran"), 0);
let n = 0;
function again() {
  if (++n < 5e6) process.nextTick(again); // now try setImmediate(again)
  else console.log("done, n =", n);
}
again();

5. A hundred thousand threads. With JDK 21 or later, save the Java example above as VT.java and run java VT.java. Change newVirtualThreadPerTaskExecutor() to newThreadPerTaskExecutor(Thread.ofPlatform().factory()) and see how far your machine gets. Save your work first, because it may struggle.

#Summary