Processes and Threads

By the end of this lesson you will be able to

• Explain what an operating system and its kernel do, why programs run in user mode, and what a system call costs.
• Say exactly what a process owns (address space, PID, file descriptors, security context, threads) and how the kernel tracks it in a process control block.
• List what threads share and what each thread keeps for itself, and explain why stacks are private but not protected.
• Describe fork, exec and wait, how Linux creates threads with clone, what each costs, and why fork in a threaded program is dangerous.
• Choose an inter-process communication method, from pipes to shared memory to TCP, and name how each one fails.
• Read process states in ps, and diagnose zombies, orphans, stuck D-state processes and containers that ignore signals.
• Write a graceful shutdown with SIGTERM, and read exit codes such as 137, 139 and 143.
• Predict the blast radius of a crash, and compare multi-process and multi-thread servers such as PostgreSQL, MySQL, Nginx and Chrome.

#Where this lesson fits

Lesson 1.2 treated a server as a long-running process, compared process-per-connection, thread-per-connection and event-loop designs in a few lines each, and walked through a rolling deploy. Lesson 1.13 showed how Apache’s MPMs, Nginx’s workers, PHP-FPM and Gunicorn use those designs. Both used the words “process” and “thread” without opening them up. This lesson opens them.

System design depends on this material more than it first appears. Backend servers, databases, web servers, network programs and any high-concurrency application are built from these parts, and most of their limits and failures are limits and failures of the operating system. This chapter covers nine topics from the course notes, across six lessons:

We keep to the operating-system view. Scheduling and switching are lesson 3.2, and data races, locks and the memory model in code are Chapter 14.

#What an operating system does

Many programs run on one machine, with one set of disks and network cards. Someone has to share them fairly, stop programs from harming each other, and hide the differences between hardware. That is the operating system. Its core is the kernel: the one program that stays in memory for as long as the machine is on and has full control of the hardware. The rest of the OS (shells, libraries, tools such as ls) are ordinary programs.

#User mode and kernel mode

The separation is enforced by the CPU, not by convention. Every modern CPU has at least two privilege levels:

• Kernel mode (ring 0 on x86, EL1 on ARM): any instruction, any memory, all hardware.
• User mode (ring 3 on x86, EL0 on ARM): ordinary computation only, on the program’s own memory.

If a program in user mode tries to touch a device, change its own memory map or halt the CPU, the hardware refuses and hands control to the kernel, which usually ends the program. So a buggy program can only hurt itself. This is also the line that Chapter 4’s hypervisors extend, and containers share it: every container on a host uses the same kernel.

#System calls

If a program cannot touch the disk or the network, how does it do anything useful? It asks. A system call is a controlled entry into the kernel: the program puts a call number and arguments in registers and runs a special instruction (syscall on x86-64), the CPU switches to kernel mode and jumps to a fixed entry point, the kernel checks everything and does the work, then returns to user mode. Linux has several hundred system calls: open, read, write, socket, accept, fork, mmap and so on. Your language’s File.read or fetch ends up there, usually through the C library.

Why the cost matters. A function call takes around a nanosecond. A cheap system call takes roughly 100 ns to 1 µs, because of the mode switch, the checks and the security mitigations added since Spectre and Meltdown. That is fast, but at millions of calls per second it dominates, which explains a lot of systems design:

• Programs buffer: one read of 64 KB instead of 64 reads of 1 KB, and logging libraries batch their writes.
• Common read-only calls such as clock_gettime are served in user mode through the vDSO, without entering the kernel.
• High-performance I/O batches calls, or avoids them altogether: epoll_wait returning many events at once, and io_uring (lesson 3.5).

#What a process is

A process is a running instance of a program. The program is a file on disk, a recipe. The process is the recipe being cooked: memory in use, files open, a position in the code. When you open Chrome, the OS creates one or more processes for it; start python twice and you have two processes running the same program.

Operating System
      |
      +---- Chrome process(es)
      |
      +---- VS Code process
      |
      +---- MySQL process
      |
      +---- Python process

Each process owns:

It also has a current directory, environment variables, signal handlers and resource limits. You can see all of this in /proc/<pid>/ on Linux, as you will under “Try it yourself”.

The kernel keeps its own record of each process, traditionally called the process control block (PCB): state, PID, parent, saved registers for when it is not running, its memory map, its file table, accounting. On Linux the structure is task_struct, and there is one per thread; the fields are shared between threads of the same process.

A typical address space, from low addresses to high:

 low   +--------------------+
       | Code (read-only)   |  the program's machine code, shareable between processes
       | Global data        |  global and static variables
       | Heap        ↓ grows|  malloc, new, every object
       |                    |
       | Shared libraries,  |
       | memory maps        |
       |                    |
       | Stack       ↑ grows|  function calls and local variables
 high  +--------------------+
       | Kernel             |  reserved for the kernel, no access from user mode
       +--------------------+

#The wall between processes

One process cannot read or write another’s memory, because it has no address that leads there. Every address goes through that process’s own page table, so the same number in another process means different physical bytes, and an address with no mapping faults at once. That wall gives you three things at the same time: fault isolation (a crash stays inside), security (one user’s process cannot read another’s secrets) and independent resources (limits, restart, upgrade).

The wall has deliberate doors. Two processes can agree to map shared memory, and a debugger with the right permission can use ptrace. The notes’ statement that a process “cannot directly access” another’s memory is true by default, not absolute.

#What a thread is

A thread is a separate flow of execution inside a process. A process always has at least one, and it can have many. Only threads run on CPUs: the scheduler in lesson 3.2 picks threads, not processes.

             A program with three threads
                    |
          +---------+---------+
          |         |         |
          v         v         v
       Thread 1  Thread 2  Thread 3
       UI        Network   Rendering

The split is the most useful thing to memorise in this chapter:

So all threads see the same objects, can use the same sockets and run the same code, but each has its own position in that code and its own local variables. Two threads can run the same function at once without their locals colliding.

Three consequences catch people out:

1. Not all memory is shared in practice, and not all of it is safe. Locals on a stack are effectively private, and registers really are. But stacks live in the one address space, so a stray pointer in one thread can overwrite another’s stack. Separate, yes; protected, no.
2. File descriptors are shared. If one thread closes a socket, it is closed for all of them. A double close can hit a descriptor that another thread has just received with the same number, which is one of the nastier bugs to find.
3. Sharing makes talking easy and getting it right hard. Threads communicate just by writing to memory, with no system call. If two threads write the same data without coordination you get a race; we name it here and teach it in Chapter 14.

#Kernel threads and user-level threads

The threads in this lesson are kernel threads: the kernel knows each one and schedules it, one-to-one. That is what pthread_create and a Java platform thread give you on Linux. Goroutines, Java virtual threads and other green threads are lighter: the language runtime multiplexes many of them onto a few kernel threads. They are covered in lesson 3.6, because they need the event loop’s machinery to work.

#Process vs thread

#Creating processes and threads

#fork, exec and wait

Unix splits “start a program” into two steps, and that split is unusually powerful.

1. fork: copy me

   fork() creates a child that is a near-copy of the parent. It returns twice: 0 in the child and the child’s PID in the parent. Memory is not actually copied at this point: parent and child share the pages through copy-on-write, and a page is copied only when someone writes to it. The mechanism is in lesson 3.4.

2. exec: become that program

   execve() replaces the current memory image with a new program. The PID, parent and open file descriptors stay, except those marked close-on-exec. Between fork and exec the child can rearrange things: that is how a shell implements > redirection and pipes, using no special support in the program it runs. On success exec never returns; if it does return, something failed, and the child must exit.

3. wait: collect the result

   wait() or waitpid() gives the parent the child’s exit status and lets the kernel delete its record. A parent that never does this creates zombies, which we come to shortly.

Windows has no fork; its CreateProcess starts a new program directly, which is one reason process creation on Windows is much slower and why Windows servers lean on threads. posix_spawn offers the same one-step style on Unix.

#Threads on Linux

Linux has no separate thread concept inside the kernel. Both are tasks, created with clone(). Flags choose what is shared: share memory, file table and signal handlers and you get a thread, whose TID is new while its PID (its thread group) is the same. That explains details you will meet: thread stacks appear as memory maps of the process, and limits such as pids.max count threads as well as processes.

#What each one costs

These are orders of magnitude on a typical Linux server. Measure on your own systems before relying on them.

Three design results follow:

• Do not create per request. Starting a thread or process for each request wastes this cost repeatedly. Reuse them with a pool (lesson 3.3) or pre-fork workers (lesson 1.13).
• fork gets slower as the process grows. Redis forks a child to write its snapshot (BGSAVE), and on a large instance that fork alone can freeze command processing for tens of milliseconds or more. Writes during the snapshot then copy pages, so a Redis server can need far more memory than its data alone.
• A thread is cheap until you have tens of thousands. A hundred thousand idle WebSocket connections (lesson 2.6) would mean a hundred thousand stacks and a lot of scheduling. That is the problem lessons 3.5 and 3.6 solve.

#Process states

A process or thread is always in one state, and on a server it is rarely “running”.

Some reading tips:

• A healthy idle server shows almost everything in S. A thread waiting on a database costs no CPU, which is why “thread count” and “CPU use” can look unrelated.
• Many threads in R with few cores means CPU saturation; you will see queueing in the load average and in latency.
• Many threads in D means storage trouble: the classic signature is high load average with low CPU (lesson 1.2 noted that Linux counts D in the load). Even SIGKILL waits until the thread wakes.

#Zombies and orphans

Two names that sound similar and get confused in interviews.

A zombie is a process that has exited but whose parent has not yet called wait. Its memory and files are gone. Only a kernel record with the exit status remains, plus its PID. Every process is a zombie for a moment, which is normal. A bug appears when a parent never reaps: ps fills with <defunct>, and eventually fork fails because the PID space (/proc/sys/kernel/pid_max) or the container’s pids.max is used up.

• You cannot kill a zombie: it is already dead. kill -9 does nothing.
• You fix the parent: handle SIGCHLD and call waitpid in a loop, or set SIGCHLD to ignored if you do not need the status. In an emergency, kill the parent, so that init adopts and reaps the zombies.

An orphan is a live process whose parent has died. It is not killed. The kernel re-parents it to PID 1 (or a nearer subreaper), and that new parent will reap it when it exits. Orphans are sometimes deliberate (nohup, daemons) and sometimes a surprise: a worker that outlives its master keeps the inherited listening socket, so the restarted service fails with address already in use.

Containers make this your problem. Your application is PID 1 in its PID namespace, so orphans in that container are re-parented to your app, which has no idea it should reap them. Use a tiny init: docker run --init, or tini as the entrypoint.

#Inter-process communication

Processes need a door in the wall. IPC is the family of those doors.

How to choose. Same machine and streams of requests: a Unix domain socket, as the sockets API is portable to TCP later. Same machine and very high rate, with locks you trust: shared memory. Anything that might ever move to another machine: TCP with a proper protocol (HTTP, gRPC) or a message broker (Chapters 30–32). In practice, a system’s microservices are simply processes doing IPC over the network, and all the extra failure modes come from that last word.

Local IPC also fails more simply. A pipe either works or reports a broken pipe; a network call can simply get no answer, leaving you unsure whether the work happened.

#Signals and graceful shutdown

A signal is a small asynchronous notification from the kernel, identified only by a number. It can arrive at any instruction. The process then runs its handler, or the default action if it has none.

Exit codes tell you what happened. When a signal kills a process, shells and container platforms report 128 + the signal number:

#Writing a graceful shutdown

Lesson 1.2 gave the deploy sequence. Here is the process’s side of it, in Node.js:

const http = require('node:http');

const server = http.createServer((req, res) => res.end('ok\n'));
server.listen(8080);

process.on('SIGTERM', () => {
  console.log('SIGTERM: draining');
  // Stop accepting; the callback fires when in-flight requests are done.
  server.close(() => {
    console.log('drained, exiting');
    process.exit(0);
  });
  // Give up shortly before the orchestrator's SIGKILL would.
  setTimeout(() => process.exit(1), 25_000).unref();
});

The same order applies in every language:

1. Fail readiness so that traffic stops, then wait a moment, because load balancers do not react instantly. In Kubernetes, endpoint removal and SIGTERM happen at the same time, not in sequence, so a few seconds of sleep in a preStop hook is common.
2. Stop accepting new work, by closing the listening socket and pausing queue consumption.
3. Finish in-flight work, with a deadline shorter than the grace period (30 seconds by default in Kubernetes, 10 in docker stop).
4. Release resources: flush logs and metrics, return unfinished messages to the queue, close database connections.
5. Exit 0.

In languages where you write the handler in native code, a signal handler may only call a short list of async-signal-safe functions, since it can interrupt anything, including malloc. So handlers just set a flag or write to a pipe, and normal code does the rest. Signals sent to a process go to any one thread that has not blocked them, so multithreaded servers usually block signals everywhere and dedicate one thread to sigwait.

Design for the case where none of this runs. A power failure, kernel panic or SIGKILL gives no warning. Graceful shutdown makes deploys clean, but only idempotent operations, transactions and a durable log make crashes safe.

#Crash isolation: who dies with whom

The rule is simple: a bug’s blast radius is the unit that shares memory with it.

If one thread dereferences a bad pointer, the kernel sends SIGSEGV and, by default, the whole process dies: all threads and all in-progress requests. It has to be that way. The bug might have overwritten anything in the shared memory, so the other threads cannot be trusted. Separate processes are different: one crashing leaves the others untouched, unless they share memory.

Three things are often mixed up here:

Security has the same shape. The 2014 Heartbleed bug in OpenSSL let attackers read a server’s process memory, private keys included, because everything lived in one address space. Splitting sensitive work into its own process, as SSH’s privilege separation does, limits what a bug can reach.

#Multi-process vs multi-thread servers

Lesson 1.2 introduced the models. Now you can see why real systems made the choices they did.

#How to choose

In practice, modern systems combine them: a few processes per machine or one per container, each with a pool of threads or an event loop, spread across many machines, with state kept outside. A container platform restarts the process when it dies, so process-level isolation is what your whole fleet relies on. The bulkhead pattern extends the same idea by giving separate dependencies separate pools (lesson 3.3).

#Common production problems

#Processes and threads in system design

• The process is your unit of failure and deployment; the machine is your unit of hardware failure. Assume any process can die at any moment, keep state outside it (lesson 1.3), and run several.
• Threads and processes cost real capacity. A thousand threads with 1 MB stacks touching 100 KB each is about 100 MB of memory plus scheduling overhead. A database with 5,000 processes is an outage waiting to happen. Put a pool in front.
• Connection limits are process and FD limits. A database’s connection cap comes from these costs. In capacity plans, count connections per app instance times the number of instances.
• Stateless processes restart well. Warm in-memory caches go with the process, so a restart can cause a cold-start spike; roll restarts slowly.
• Isolation is a dial: thread → process → container → virtual machine → separate machine. Each step costs more and contains more (Chapter 4 covers the far end).
• Interview angles: “Process vs thread?” Answer with what is shared, the blast radius, and the cost. “Why does PostgreSQL need PgBouncer and MySQL mostly not?” “What is a zombie, and can you kill one?” “Why does a container take 30 s to stop?” “What does exit 137 mean?” “Would you run a plugin in a thread or a process?” “Why can one bad pointer take down 200 requests?”

#Try it yourself

On a Linux machine, or in a container with procps and python3:

# Look at processes: pid, parent, state, thread count, resident KB, name
ps -eo pid,ppid,stat,nlwp,rss,comm --sort=-nlwp | head

# Every thread of every process (LWP is the TID)
ps -eLf | head

# Everything the kernel knows about your shell
grep -E 'Name|State|Tgid|^Pid|PPid|Threads|Uid' /proc/$$/status
ls -l /proc/$$/fd        # its file descriptors
id                       # its security context

# Watch the system calls behind a command, and count them
strace -f -e trace=clone,clone3,execve,wait4 bash -c 'ls > /dev/null'
strace -c ls > /dev/null

# Make a zombie for 30 seconds, then see it
python3 -c "import os, time; pid = os.fork(); (os._exit(0) if pid == 0 else time.sleep(30))" &
sleep 1; ps -o pid,ppid,stat,comm --ppid $!

# Read exit codes: 143 and 137
sleep 100 & P=$!; kill -TERM $P; wait $P; echo "exit code $?"
sleep 100 & P=$!; kill -KILL $P; wait $P; echo "exit code $?"

# One thread segfaults and the whole process goes down: the print never runs, exit code 139
python3 -c 'import ctypes, threading, time
threading.Thread(target=lambda: ctypes.string_at(0)).start()
time.sleep(5); print("never printed")'; echo "exit code $?"

Threads share memory, processes do not. Save this as share.py and run it:

import multiprocessing as mp
import threading

counter = 0

def bump():
    global counter
    counter = 100

if __name__ == '__main__':
    t = threading.Thread(target=bump)
    t.start(); t.join()
    print('after thread :', counter)   # 100: same memory

    counter = 0
    p = mp.Process(target=bump)
    p.start(); p.join()
    print('after process:', counter)   # 0: the child changed its own copy

A Dockerfile that lets signals reach your app:

# Shell form runs under sh, so the app does not get SIGTERM:
#   CMD node server.js
# Exec form makes the app PID 1, and it needs its own handler:
CMD ["node", "server.js"]

Run it with docker run --init to get signal forwarding and zombie reaping, then time docker stop <name>. If that takes exactly 10 seconds, the signal is not getting through.

#Summary

Key takeaways

• The kernel is the privileged core of the OS. Programs run in user mode and ask for everything through system calls, which cost roughly 100 ns to 1 µs, so programs buffer and batch.
• A process is a running program with its own virtual address space, PID, FDs, security context and threads, tracked in the kernel’s PCB. A wall separates processes, with deliberate doors.
• A thread is a flow of execution in a process. Threads share heap, globals, code and FDs; each owns a stack, program counter, registers and TID. Stacks are private but not protected.
• fork copies with copy-on-write, exec loads a new program, wait collects the status. On Linux, threads are tasks from clone that share everything. Creation is expensive enough that servers reuse workers. Do not fork and carry on in a threaded program.
• States: Ready and Running are both R, S waits cheaply, D cannot be interrupted, and high load with an idle CPU means storage trouble.
• A zombie is dead but unreaped and cannot be killed; fix the parent. An orphan is alive and adopted by PID 1. In a container, PID 1 is your app, so use an init.
• IPC: pipes for streams, Unix sockets locally, shared memory for speed at the price of shared failure, TCP for anything that may cross machines.
• SIGTERM can be caught for graceful shutdown; SIGKILL cannot. Exit code = 128 + signal: 137, 139, 143. Shell form in a Dockerfile, or PID 1 with no handler, swallows SIGTERM.
• A bug’s blast radius is whoever shares its memory: a segfault kills all threads of a process. PostgreSQL resets all backends on one crash because of shared buffers, MySQL loses the whole process, and Chrome isolates sites.
• Real systems combine a few processes, pools of threads or event loops, and many machines, with state kept outside.

Self-check: Processes and threads33 questions