Web Servers and App Servers: Nginx, Apache, Caddy and the Rest

By the end of this lesson you will be able to

• Say exactly what a web server does with each request, and tell a web server, an application server, a reverse proxy and a load balancer apart, even when one program plays all four roles.
• Explain how Apache works inside (virtual hosts, modules, .htaccess, the prefork, worker and event MPMs) and how Nginx works inside (master and workers, event loops, server and location blocks).
• Read and write a real Nginx config: static files, try_files, reverse proxy headers, load balancing, caching, rate limiting, TLS, and a graceful reload.
• Place Caddy, IIS, LiteSpeed, Tomcat, HAProxy, Envoy, Traefik and Varnish correctly in a request path.
• Explain why application servers exist and how the web server talks to them: CGI, FastCGI and PHP-FPM, WSGI and ASGI, Rack, servlets, Kestrel, Node.js.
• Diagnose 502, 504, 499, 503, 413, 403 and redirect loops from the logs, and describe a production stack in a system design interview.

#What a web server actually does

The lesson What is a server? showed that a server is a program that listens on a port and answers clients. A web server is that idea specialised for one protocol: it is software that speaks HTTP. Every request it receives goes through the same five stages:

1. Accept the connection. The server is listening on port 80 (HTTP) and 443 (HTTPS). The operating system completes the TCP handshake and hands the new connection over. On 443 the server then performs the TLS handshake with its certificate and private key.
2. Parse the request. It reads the request line and headers (GET /images/logo.png HTTP/1.1, Host: example.com, …), and the body if there is one. Badly formed requests are rejected here with 400.
3. Decide what to do. Using its configuration, it matches the Host header to a site and the path to a rule, and picks one action: serve a file from disk, hand the request to an application, redirect (301/302), or reject it (403, 404, 413, 429…).
4. Send the response. Status line, headers (Content-Type, Content-Length, Cache-Control…) and the body, possibly compressed. With keep-alive the connection then stays open for the next request.
5. Log it. One line in the access log (who, what, status, size, how long) and, if something went wrong, a line in the error log.

That third stage is where all the interesting differences live. Content splits into two kinds:

• Static content is a file that already exists and is the same for everyone: HTML pages built in advance, CSS, JavaScript, images, fonts, videos, downloads. The web server reads it from disk (or memory) and sends it. This is very fast and needs no programming language at all.
• Dynamic content is built per request by code: your feed, a shopping cart, a search result, an API response in JSON. The web server cannot produce it alone. It passes the request to an application written in PHP, Python, Ruby, Java, C#, JavaScript or Go, which usually talks to a database, and relays the answer back.

Play the scenarios below: one request for a static file, one for a dynamic page, and the everyday ways each one fails.

#Four roles, often one program

People use four words for the boxes in front of an application, and the same software often plays several of them at once:

A single Nginx process can serve /static/ from disk (web server), forward /api/ to three app servers (reverse proxy and load balancer) and cache the answers. Meanwhile Tomcat is called an application server but also speaks HTTP perfectly well, so it is a web server too. Name the role a box plays in your design, not the brand.

#A short history: from one office computer to the C10K problem

• 1990, CERN httpd. Tim Berners-Lee wrote the first web server at CERN, alongside the first browser.
• 1993, NCSA httpd. Written at NCSA (the lab that also made the Mosaic browser). It introduced CGI, the first standard way for a web server to run a program and return its output.
• 1995, Apache. When NCSA httpd development stalled, a group of webmasters collected their fixes (“patches”) and released them together as the Apache HTTP Server. Within about a year it was the most used web server in the world, and stayed there for roughly two decades. Its strength was a module system: anyone could add features.
• 1999, the C10K problem. Dan Kegel asked how one server could handle ten thousand concurrent clients. Servers that dedicated one process or thread to each connection ran out of memory long before that. The answer was event-driven, non-blocking I/O: a few threads, each watching thousands of sockets through the operating system’s readiness notifications (epoll on Linux, kqueue on FreeBSD and macOS).
• 2004, Nginx. Igor Sysoev built Nginx (pronounced “engine-x”) to serve the heavily loaded Russian portal Rambler, designed from day one around that event-driven model. It spread quickly as the fast front for static files and reverse proxying. A company formed around it in 2011 and F5 bought it in 2019; Nginx remains open source, and a core developer started a community fork, FreeNginx, in 2024.
• 2015, Caddy. Written in Go by Matt Holt, Caddy was the first popular server to get HTTPS certificates automatically for every site, right after Let’s Encrypt began issuing free certificates.

Today the picture roughly looks like this, and it shifts every year: Nginx and Apache each run a very large share of websites, with Nginx ahead; Cloudflare shows up for a big slice because it answers in front of millions of sites; LiteSpeed is a strong fourth thanks to WordPress hosting; Caddy, IIS, Node.js and others make up the rest. Survey numbers differ because some count every site (including millions of parked domains) and others only the busiest ones.

#Apache HTTP Server in depth

Apache (the program is called httpd, or apache2 on Debian and Ubuntu) is flexible, extremely well documented and still the default on much shared hosting.

#Configuration: virtual hosts, directories and modules

The main file is httpd.conf (Red Hat family) or apache2.conf (Debian family). Debian splits the rest into folders: one file per site in sites-available/, switched on with a2ensite (which creates a link in sites-enabled/), and modules switched on with a2enmod.

The building blocks:

• VirtualHost blocks: one block per site. Apache picks the block by IP, port and the Host header (ServerName, ServerAlias), so one server can host hundreds of domains.
• DocumentRoot: the folder whose files are served for that site.
• Directory blocks (and Location, Files): rules that apply to a part of the file system or URL space: who may access it, whether .htaccess files are read, whether directory listings are shown.
• Modules add almost every feature: mod_ssl (TLS), mod_rewrite (rewriting and redirecting URLs with regular expressions), mod_proxy with mod_proxy_http or mod_proxy_fcgi (reverse proxy to app servers), mod_headers (add or remove headers), mod_deflate (gzip), mod_php (runs PHP inside each Apache process).

A real site: HTTP redirects to HTTPS, files come from public/, and /api/ goes to an app on port 8000.

# /etc/apache2/sites-available/example.conf
<VirtualHost *:80>
    ServerName example.com
    ServerAlias www.example.com
    Redirect permanent / https://example.com/
</VirtualHost>

<VirtualHost *:443>
    ServerName example.com
    DocumentRoot /var/www/example/public

    SSLEngine on
    SSLCertificateFile    /etc/letsencrypt/live/example.com/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem

    <Directory /var/www/example/public>
        Options -Indexes +FollowSymLinks   # no directory listings
        AllowOverride None                 # ignore .htaccess (faster)
        Require all granted
    </Directory>

    ProxyPass        /api/ http://127.0.0.1:8000/
    ProxyPassReverse /api/ http://127.0.0.1:8000/
    RequestHeader set X-Forwarded-Proto "https"

    Header always set X-Content-Type-Options "nosniff"
    ErrorLog  ${APACHE_LOG_DIR}/example-error.log
    CustomLog ${APACHE_LOG_DIR}/example-access.log combined
</VirtualHost>

sudo a2enmod ssl proxy proxy_http headers
sudo a2ensite example
sudo apachectl configtest          # "Syntax OK" before touching the live server
sudo systemctl reload apache2      # graceful: running requests finish first

#.htaccess: per-directory overrides

An .htaccess file is a small config file placed inside a content folder. If the Directory block allows it (AllowOverride All or a list of allowed directive types), Apache applies the directives in it to that folder and everything below: rewrite rules, redirects, password protection, custom error pages, cache headers.

Why it exists: on shared hosting, hundreds of customers share one Apache and none of them may edit the main config. With .htaccess each customer controls their own site by uploading a file, and changes apply on the next request without a reload. WordPress, for example, writes its “pretty permalinks” rewrite rules there.

Why it costs performance: Apache does not cache those files. For every request it must look for .htaccess in the target folder and every parent folder up to the root, read and parse any it finds. A request for /blog/2026/10/post.html can mean several extra file-system lookups each time.

#MPMs: how Apache handles many connections

Apache’s connection handling is itself a pluggable module, the MPM (Multi-Processing Module). You load exactly one:

The key limit is MaxRequestWorkers (called MaxClients before Apache 2.4): the maximum number of requests served at the same time. In prefork that is the number of processes; in worker and event it is processes times ThreadsPerChild. When all are busy, new connections wait in the kernel’s listen queue and eventually time out. Apache logs a warning that the server reached MaxRequestWorkers when this happens.

The animation shows why this matters: ten fast clients are easy for anyone; a thousand slow clients sitting on keep-alive connections are not.

#Nginx in depth

Nginx was designed around one goal: many connections, little memory, no surprises under load. It is the most common reverse proxy and static file server on the internet.

#Master and workers

When you start Nginx, you get one master process and several worker processes:

• The master runs as root. It reads and checks the configuration, opens the log files and binds the listening sockets on ports 80 and 443 (ports below 1024 need root on Linux). Then it starts the workers and supervises them. It never handles a client itself.
• The workers inherit the open listening sockets, then switch to an unprivileged user (user www-data; or user nginx;). Usually there is one worker per core (worker_processes auto;). Every client connection is handled by one worker from start to finish.

If a worker crashes, the master starts a new one. If an attacker found a bug in request parsing, they would land in a process with no root rights. The same split makes reloads graceful, as you will see below.

#The event loop: why memory stays flat

Each worker is a single thread running an event loop. It never waits on one client. Instead, it asks the kernel (epoll on Linux, kqueue on FreeBSD and macOS) “tell me which of my thousands of sockets are ready”, then does a small piece of work on each ready socket (read some bytes, parse a header, write a chunk of the response) and goes back to waiting. All socket operations are non-blocking: if a socket has no data yet, the call returns at once and the worker moves on.

A connection that is idle (a keep-alive client thinking, a slow mobile client trickling bytes) costs Nginx only a small structure and some buffer memory, on the order of kilobytes, not a whole process or thread. That is why ten thousand idle connections barely change Nginx’s memory, and why it shrugs off slow clients that would exhaust a prefork Apache. The limit per worker is worker_connections (often set to 1024 or more); the theoretical total is workers × worker_connections, and a proxied request uses two connections (client side and upstream side).

#Config structure

Nginx configuration (/etc/nginx/nginx.conf, often with include files in conf.d/ or sites-enabled/) is a tree of contexts (blocks) holding directives that end in ;:

Directives set in an outer block are inherited by inner blocks unless overridden. Here is a complete, realistic configuration for one site:

user  www-data;
worker_processes  auto;
error_log  /var/log/nginx/error.log warn;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile      on;
    keepalive_timeout  65s;
    server_tokens off;                 # hide the version number
    client_max_body_size 20m;          # default is 1m

    log_format main '$remote_addr "$request" $status $body_bytes_sent '
                    'rt=$request_time urt=$upstream_response_time';
    access_log /var/log/nginx/access.log main;

    gzip on;
    gzip_types text/css application/javascript application/json image/svg+xml;

    upstream app {
        least_conn;
        server 10.0.1.11:8000 max_fails=3 fail_timeout=10s;
        server 10.0.1.12:8000 max_fails=3 fail_timeout=10s;
        server 10.0.1.13:8000 backup;
        keepalive 32;                  # reuse connections to the app
    }

    proxy_cache_path /var/cache/nginx keys_zone=pages:10m max_size=1g inactive=10m;
    limit_req_zone $binary_remote_addr zone=login:10m rate=5r/s;

    server {                           # HTTP: only redirect
        listen 80;
        server_name example.com www.example.com;
        return 301 https://example.com$request_uri;
    }

    server {
        listen 443 ssl;
        http2 on;                      # older versions: listen 443 ssl http2;
        server_name example.com;
        ssl_certificate     /etc/letsencrypt/live/example.com/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
        ssl_protocols TLSv1.2 TLSv1.3;

        root /var/www/example/public;

        location /assets/ {            # fingerprinted files: cache for a year
            expires 1y;
            add_header Cache-Control "public, immutable";
        }

        location = /login {
            limit_req zone=login burst=10 nodelay;
            proxy_pass http://app;
        }

        location /api/ {
            proxy_pass http://app;
            proxy_http_version 1.1;
            proxy_set_header Connection "";
            proxy_set_header Host              $host;
            proxy_set_header X-Real-IP         $remote_addr;
            proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_connect_timeout 5s;
            proxy_read_timeout    30s;
            proxy_cache pages;
            proxy_cache_valid 200 1m;
            proxy_cache_use_stale error timeout updating;
        }

        location / {
            try_files $uri $uri/ /index.html;   # single-page app fallback
        }
    }
}

#How Nginx picks a location

Inside a server, exactly one location handles a request. The order is precise and a favourite interview question:

1. Exact match location = /login: if the path is exactly this, stop here.
2. Otherwise Nginx finds the longest matching prefix among normal prefix locations (location /api/, location /) and remembers it. If that longest prefix is marked ^~ (location ^~ /assets/), it is used immediately and regular expressions are skipped.
3. Otherwise Nginx tries the regular expression locations (~ case-sensitive, ~* case-insensitive) in the order they appear in the file. The first one that matches wins.
4. If no regular expression matches, the remembered longest prefix is used.

So location ~* \.(png|jpg)$ beats location /images/ for /images/a.png, unless you write location ^~ /images/. Order in the file matters only among regular expressions.

#Serving static files

• root maps a URL to a path: with root /var/www/site; the URL /css/app.css becomes /var/www/site/css/app.css. (alias replaces the matched part of the path instead of appending it; mixing them up is a classic cause of 404.)
• try_files $uri $uri/ /index.html; tries each option in turn: the file, then a folder with an index file, then a fallback. The fallback can be a file (for single-page apps) or a named location that proxies to the app (the usual PHP and Rails pattern: “serve the file if it exists, otherwise ask the app”).
• sendfile on; lets the kernel copy file data straight from the page cache to the socket, without copying it through Nginx’s memory (“zero-copy”). Combined with tcp_nopush, headers and the start of the file go out in full packets.
• gzip on; (or the separate brotli module) compresses text types on the fly; gzip_static on; serves a pre-compressed .gz file if it exists, which saves CPU.
• Cache headers: expires and add_header Cache-Control tell browsers and CDNs how long to keep a file. Fingerprinted files (app.3f9a1c.js) can be cached for a year; HTML usually not.

#Reverse proxy

proxy_pass http://app; forwards the request to an upstream group or a single address (http://127.0.0.1:3000, or a Unix socket http://unix:/run/app.sock). Four things you almost always configure:

• Headers. The app sees the connection coming from Nginx, so Nginx passes what it knows: Host (the original host name), X-Real-IP and X-Forwarded-For (the client IP chain), X-Forwarded-Proto (https if the client used TLS). The app must trust these only from the proxy.
• Timeouts. proxy_connect_timeout (establishing the connection), proxy_send_timeout and proxy_read_timeout (the time between two successive reads from the app, 60 s by default). When the read timeout expires, the client gets 504.
• Buffering. By default Nginx reads the app’s response into buffers (spilling to a temporary file if large) as fast as the app can send, then trickles it to the client at the client’s speed. The expensive app worker is freed immediately instead of waiting for a slow phone. Turn it off (proxy_buffering off;) for streaming responses such as server-sent events. Request bodies are buffered too, so the app receives an upload only once it has fully arrived.
• Upgrades. WebSockets need proxy_http_version 1.1 and the Upgrade and Connection headers passed through.

#Load balancing

An upstream block lists servers and a method:

Open-source Nginx uses passive health checks: if connecting to a server or reading from it fails max_fails times (default 1) within fail_timeout (default 10 s), it is marked unavailable for fail_timeout, then tried again. The request that failed is retried on the next server (proxy_next_upstream, by default only for errors and timeouts on safe attempts). Active health checks (probing /health on a schedule) are in the commercial Nginx Plus, HAProxy, Envoy and cloud load balancers.

#Caching and rate limiting

• proxy_cache stores upstream responses on disk with an index in shared memory (keys_zone). Later requests for the same key (by default scheme, host and URI) are answered from the cache without touching the app. proxy_cache_use_stale serves an old copy when the app is down or slow, and proxy_cache_lock lets only one request refill an expired entry while the others wait, which protects the app from a stampede. Responses that set cookies or say Cache-Control: private are not cached by default.
• limit_req uses a “leaky bucket” per key (usually the client IP): rate=5r/s with burst=10 allows short bursts, then rejects the excess. The default rejection status is 503; many teams set limit_req_status 429; so clients see the correct “Too Many Requests”. limit_conn caps simultaneous connections per key.

#TLS, HTTP/2 and HTTP/3

ssl_certificate and ssl_certificate_key point to the certificate chain and private key (Nginx needs the key file readable only by root, which the master opens before dropping privileges). ssl_protocols TLSv1.2 TLSv1.3; disables old versions. HTTP/2 is switched on with http2 on; (older versions: listen 443 ssl http2;). HTTP/3 over QUIC is supported since version 1.25 with listen 443 quic; plus an Alt-Svc header announcing it, because browsers first connect over TCP and then upgrade. Nginx does not fetch certificates itself; a tool such as certbot obtains them and reloads Nginx.

#Changing config safely: test and graceful reload

sudo nginx -t            # parse and check every file; prints the line of any error
sudo nginx -s reload     # or: systemctl reload nginx (sends a HUP signal to the master)

On reload the master re-reads the configuration. If it is valid, it opens any new sockets, starts new workers with the new config and tells the old workers to stop accepting new connections; they finish the requests already in progress and exit. No connection is dropped. If the new config is invalid, the master logs the error and keeps running the old configuration. Running nginx -t first is still the habit, because it shows you the error immediately, and a restart (stop then start) with a broken file would leave you with no web server at all. Long-lived connections such as WebSockets keep old workers alive; worker_shutdown_timeout caps how long.

#Nginx cannot run PHP: FastCGI to PHP-FPM

Nginx has no language runtime inside. For PHP it speaks FastCGI to a separate pool of PHP processes, PHP-FPM, usually over a Unix socket:

location / {
    try_files $uri $uri/ /index.php?$args;
}
location ~ \.php$ {
    include fastcgi_params;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_pass unix:/run/php/php8.3-fpm.sock;
}

The same idea applies to every language: Nginx serves files and proxies everything else to an app server (by HTTP, FastCGI or uwsgi protocol).

#OpenResty and Nginx in Kubernetes

OpenResty is Nginx bundled with a fast Lua engine, so you can write request logic in Lua inside the event loop: custom authentication, dynamic routing, rate limits based on a database. Kong and other API gateways are built on it. In Kubernetes, Nginx long served as the most common ingress controller, the reverse proxy that routes outside traffic to services by host name and path. The community ingress-nginx project was retired by the Kubernetes project in 2026; new clusters tend to use the Gateway API with controllers built on Envoy, Nginx (F5’s own controller), Traefik or HAProxy.

#Apache vs Nginx

A common combination is Nginx in front of Apache: Nginx terminates TLS, serves static files, absorbs slow clients and buffers, and proxies only dynamic requests to Apache listening on 127.0.0.1:8080, where .htaccess and mod_php keep working for an existing PHP application. Apache’s mod_remoteip then restores the real client IP from X-Forwarded-For.

#Other web servers

#Caddy: HTTPS by default

Caddy is a single Go binary with a short config file, the Caddyfile. Its signature feature is automatic HTTPS: for every site name in the config, Caddy obtains a certificate from Let’s Encrypt (with another free certificate authority as fallback) using the ACME protocol, redirects HTTP to HTTPS, and renews certificates before they expire. HTTP/2 and HTTP/3 are on by default. A complete reverse proxy with TLS:

example.com {
    encode gzip zstd
    root * /var/www/example/public
    reverse_proxy /api/* localhost:8000
    file_server
}

How the certificate arrives: the certificate authority must verify that you control the name. With the HTTP-01 challenge it gives Caddy a token, and then fetches http://example.com/.well-known/acme-challenge/<token>; only the real server for that name can answer. So DNS must already point to the server and port 80 must be reachable from the internet. The alternatives are TLS-ALPN-01 (proved on port 443) and DNS-01 (proved by creating a TXT record through your DNS provider’s API, the only way to get wildcard certificates or certificates for machines not reachable from outside). Let’s Encrypt certificates currently last 90 days, with shorter lifetimes being introduced; Caddy renews when about a third of the lifetime remains, roughly 30 days before expiry for a 90-day certificate.

#The rest of the field

• Microsoft IIS (Internet Information Services) is the web server built into Windows Server. A kernel driver (http.sys) receives requests and hands them to application pools, each a set of worker processes (w3wp.exe) that can be recycled independently. It is the traditional home of ASP.NET applications and is configured through a graphical manager or web.config files (its counterpart to .htaccess).
• LiteSpeed Web Server (commercial) is event-driven like Nginx but reads Apache’s configuration and .htaccess files directly, so hosting companies can swap Apache out without customers changing anything. Its free sibling OpenLiteSpeed has its own config format but still understands .htaccess rewrite rules. Its built-in page cache, LSCache, has a popular WordPress plugin, which is why it is common on WordPress hosting.
• Lighttpd (“lighty”, 2003) was written as an answer to the C10K problem: event-driven and tiny. Today it appears mostly on embedded devices and routers.
• Apache Tomcat is a Java servlet container (see below) that is also a capable HTTP server on its own; Jetty and Undertow likewise.
• Language built-ins. Node’s http module and Go’s net/http are real production servers: the application itself listens on a port. Python’s python -m http.server and PHP’s php -S are for development only; their documentation says so (no security hardening, little concurrency).

#Application servers: where your code actually runs

A web server like Nginx knows nothing about Python, Ruby, Java or PHP. Something must keep the language runtime loaded, keep your application code and its database connections warm in memory, and run a request through it. That is the application server. Between the two sits an interface: an agreed way to hand over a request and get a response back. They appeared roughly in this order.

#CGI: a new process for every request

With CGI (Common Gateway Interface, 1993), the web server starts a new process for each request, passes the request details in environment variables (REQUEST_METHOD, QUERY_STRING…) and the body on standard input, and sends whatever the program prints as the response. Simple and language-neutral, but starting an interpreter, loading the code and opening a database connection on every request is slow, and a burst of traffic means a burst of processes. CGI survives only in tiny embedded tools.

#FastCGI and PHP-FPM: keep the processes alive

FastCGI keeps a pool of long-running processes. The web server sends each request to a free one over a socket using a binary protocol, and the process stays alive for the next request. PHP-FPM (FastCGI Process Manager) is the standard way to run PHP today, behind Nginx, Apache (event MPM with mod_proxy_fcgi) or Caddy. Its pool settings matter in production:

• pm = static keeps exactly pm.max_children processes; pm = dynamic keeps between pm.min_spare_servers and pm.max_spare_servers idle, up to pm.max_children; pm = ondemand starts processes only when requests arrive and stops them after an idle timeout (saves memory on quiet sites).
• pm.max_children is the hard limit on simultaneous PHP requests. Size it from memory: free RAM divided by the memory of one PHP process. When all children are busy, new requests queue on the socket; the log says server reached pm.max_children setting, users see slow pages, then Nginx answers 504 (or 502 if the queue overflows).
• pm.max_requests recycles a process after that many requests, a safety net against memory leaks; request_terminate_timeout kills a request that runs too long.

#Python: WSGI and ASGI

• WSGI (Web Server Gateway Interface) is Python’s standard: the app is a callable that takes the request environment and returns the response. One request occupies one worker until it finishes. Frameworks: Django, Flask. Servers: Gunicorn, a pre-fork server whose master starts N worker processes (a common starting point is 2 × cores + 1) and restarts any that crash or exceed --timeout (30 s by default); and uWSGI, very configurable but now in maintenance mode.
• ASGI (Asynchronous Server Gateway Interface) is the async successor: one worker can interleave thousands of requests while they wait on I/O, and it supports WebSockets and long-lived connections. Frameworks: FastAPI, Starlette, Django (async views). Servers: Uvicorn, Hypercorn, Daphne. In production Uvicorn is often run with several worker processes (its own --workers option, or under Gunicorn with a Uvicorn worker class).

#Ruby: Rack, Puma and Unicorn

Rack is Ruby’s interface (Rails, Sinatra). Puma, the Rails default, combines threads inside each process with several worker processes (cluster mode), so it can use all cores and also overlap I/O waits. Unicorn is a strict pre-fork server, one request per process, designed to sit behind Nginx, which shields it from slow clients.

#Java: servlet containers and application servers

Java web apps implement the Servlet API. A servlet container (Tomcat, Jetty, Undertow) runs inside one long-lived JVM, accepts HTTP itself and calls your servlets on a thread pool. Apps used to be packaged as a WAR file and deployed into a running Tomcat; today Spring Boot usually embeds Tomcat (or Jetty, Undertow) inside a runnable JAR, so “the app is the server”. A full application server (Jakarta EE, formerly Java EE: WildFly / JBoss, WebLogic, WebSphere, GlassFish) adds transactions, messaging, connection pools and more enterprise services on top of the servlet container.

#.NET, Node.js and Go

• .NET: ASP.NET Core apps run on Kestrel, a fast cross-platform server built into the framework. It is commonly placed behind IIS on Windows, or Nginx or a cloud load balancer on Linux, for TLS, static files and process management.
• Node.js (and Next.js, Express, NestJS): the application is the server. Your code calls listen(3000) and runs on one event loop, on one core. To use every core, run several processes: Node’s cluster module, PM2 in cluster mode (which also restarts crashes and reloads without downtime), or several containers behind a load balancer.
• Go: one compiled binary with net/http built in; nothing else to install. It is often still placed behind a load balancer for TLS, routing and running several copies.

#Other software in the request path

Web servers share the path with more specialised tools:

• HAProxy (High Availability Proxy, since 2000) is a dedicated load balancer for TCP (layer 4) and HTTP (layer 7). It does not serve files; it balances, and it is extremely fast and precise at it: active health checks, connection queues per server, stick tables for rate limits and stickiness, detailed statistics. Its PROXY protocol passes the client IP over plain TCP.
• Envoy (created at Lyft, now a Cloud Native Computing Foundation project) is a modern proxy built for microservices. Its configuration can be changed at runtime through APIs called xDS, without reloads, which is why service meshes such as Istio run an Envoy sidecar next to every service: it handles retries, timeouts, mutual TLS and metrics for each call. Many API gateways and Kubernetes Gateway controllers are built on it.
• Traefik is a reverse proxy that discovers services by itself: it watches Docker labels or Kubernetes resources and creates routes as containers start and stop, and gets certificates automatically through ACME. Popular for small clusters and home labs.
• Varnish is a pure HTTP cache (“web accelerator”) placed in front of a web server, with its own configuration language for caching rules. It does not terminate TLS itself, so it usually sits behind Nginx, HAProxy or a dedicated TLS proxy.
• Managed equivalents do the same jobs as a service: AWS ALB (HTTP, layer 7: path and host routing, WebSockets, gRPC) and NLB (TCP/UDP, layer 4, static IPs, very high throughput), CloudFront and Cloudflare (CDN, edge TLS, caching, WAF), Google Cloud Load Balancing. You do not patch or scale them, but you configure the same ideas: listeners, target groups, health checks, timeouts.

Kubernetes ingress controllers are the same idea adapted to a cluster. Inside a cluster, pods come and go, so you cannot write their IPs into a config file. An ingress controller (or, newer, a Gateway API controller) is a reverse proxy (Nginx, Envoy, Traefik, HAProxy, or a cloud load balancer) that watches the cluster and rewrites its own routing whenever services change. Traffic flows: cloud load balancer → ingress controller pods → Service → application pods.

#Typical production stacks

That last row is how this very site runs: OpenNext packages the Next.js app for AWS Lambda behind CloudFront. There is no Nginx anywhere; CloudFront terminates TLS and caches, S3 serves the static files, and Lambda runs the app code per request.

#When things go wrong

Most incidents in this layer show up as a handful of status codes. Learn who generates each one and you know where to look.

#Reading the logs

The access log tells you what happened, the error log tells you why. With the log format from the config above:

# /var/log/nginx/access.log
203.0.113.7 "GET /api/orders HTTP/2.0" 200 1532 rt=0.084 urt=0.081
203.0.113.7 "GET /assets/app.3f9a1c.js HTTP/2.0" 200 48211 rt=0.002 urt=-
198.51.100.4 "POST /api/upload HTTP/2.0" 413 183 rt=0.000 urt=-
192.0.2.55 "GET /api/report HTTP/2.0" 504 167 rt=30.001 urt=30.001

# /var/log/nginx/error.log
connect() failed (111: Connection refused) while connecting to upstream, upstream: "http://10.0.1.12:8000/api/orders"
upstream timed out (110: Connection timed out) while reading response header from upstream
client intended to send too large body: 52428800 bytes
connect() to unix:/run/php/php8.3-fpm.sock failed (13: Permission denied) while connecting to upstream

urt=- means no upstream was involved (static file or rejected early). A request where rt is much larger than urt spent its time talking to a slow client, not waiting on the app. Useful habits: tail -f both files while reproducing a problem, count status codes with awk (see Try it yourself below), and grep upstream error.log to see every failed upstream call.

#Slow clients and slowloris

A slowloris attack opens many connections and sends request headers extremely slowly, a few bytes every few seconds, never finishing. A server with one process or thread per connection (Apache prefork, and to a lesser degree worker) runs out of workers with a few hundred such connections, using almost no attacker bandwidth. An event-driven server (Nginx, Apache event for idle keep-alives, Caddy, HAProxy) keeps each waiting connection as a small structure, so the attack costs it little; timeouts (client_header_timeout, client_body_timeout, Apache’s mod_reqtimeout) and per-IP connection limits close the rest. The same mechanism explains why a proxy in front helps every app server: it absorbs the slow part.

#Performance and security checklist

Performance

• Keep-alive on for clients (keepalive_timeout) and to upstreams (keepalive in the upstream block plus proxy_http_version 1.1), so connections and TLS handshakes are reused.
• Compression: gzip or brotli for text types; pre-compressed files with gzip_static.
• Caching headers: long Cache-Control with immutable for fingerprinted assets, short or no-cache for HTML; a CDN in front for static files.
• sendfile and tcp_nopush for static files; open_file_cache for busy sites with many small files.
• HTTP/2 everywhere, HTTP/3 where clients benefit (mobile, lossy networks).
• TLS: TLS 1.2 and 1.3 only, modern ciphers, session resumption, HSTS once HTTPS works everywhere.
• Right-sized pools: worker_processes auto, file descriptor limits, app workers sized by memory and CPU, measured with a load test.

Security

• Unprivileged workers: the master alone runs as root; app servers run as their own user.
• Hide versions: server_tokens off, ServerTokens Prod, remove X-Powered-By from the app.
• Request limits: client_max_body_size, header size limits, limit_req on login and expensive endpoints, limit_conn per IP.
• Timeouts: short header and body timeouts against slow clients; nested upstream timeouts.
• No directory listings (autoindex off, Options -Indexes) and deny hidden files (location ~ /\. { deny all; }, but allow /.well-known/).
• Security headers: X-Content-Type-Options, Content-Security-Policy, Referrer-Policy, HSTS.
• Trust forwarded headers only from your proxies (Nginx set_real_ip_from, Apache mod_remoteip).
• Keep it updated, and test every config change (nginx -t, apachectl configtest) before a graceful reload.

#Web and app servers in system design

In an interview, draw the request path and name each box by its role:

Client ─► DNS ─► CDN (static files, edge TLS, cache)
       ─► Load balancer (TLS termination, health checks)       ┐ stateless,
       ─► Web tier: Nginx (static, buffering, limits, routing) │ scale out
       ─► App tier: app server workers running your code       ┘ behind the LB
       ─► Cache (Redis) and database (primary + replicas)

Points worth saying out loud:

• Keep the web and app tiers stateless (sessions in Redis or tokens, files in object storage), so you can add or remove machines behind the load balancer freely. That is horizontal scaling: more identical app servers, not a bigger one.
• Concurrency per box is set by the app server’s workers. A rough capacity estimate is workers × machines ÷ average request time: 8 workers at 100 ms each serve about 80 requests per second per machine. If requests mostly wait on I/O, async servers or more threads raise that number; if they burn CPU, only more cores help.
• The front absorbs what the app should not see: slow clients, TLS, static files, abusive traffic. This is why the web tier needs far fewer machines than the app tier.
• Remove single points of failure: at least two load balancer nodes (or a managed one across zones), at least two app servers, health checks that test real dependencies, and graceful shutdown so deploys drain connections instead of cutting them.
• Deploys: rolling (replace instances a few at a time), blue-green (switch the load balancer to a new fleet) or canary (send a small weight to the new version first). All rely on the load balancer and health checks you configured.

#Try it yourself

Run Nginx in Docker and look at its response headers:

docker run --rm -d --name web -p 8080:80 nginx
curl -I http://localhost:8080
#   HTTP/1.1 200 OK
#   Server: nginx/1.x.y         <- the version leaks by default
#   Content-Type: text/html
docker exec web nginx -T | head -40     # print the full active config
docker logs -f web                       # access log goes to stdout in the image

A minimal config that serves a folder and proxies /api/ to an app on port 3000:

# site.conf
server {
    listen 80;
    root /usr/share/nginx/html;
    location / {
        try_files $uri $uri/ =404;
    }
    location /api/ {
        proxy_pass http://host.docker.internal:3000;   # the host machine (Linux: add --add-host=host.docker.internal:host-gateway)
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

docker run --rm -d --name web2 -p 8081:80 \
  -v "$PWD/site.conf:/etc/nginx/conf.d/default.conf:ro" nginx
docker exec web2 nginx -t          # test the config
docker exec web2 nginx -s reload   # graceful reload after editing site.conf

The same reverse proxy with Caddy, HTTPS included (for localhost, Caddy uses its own local certificate authority):

localhost {
    reverse_proxy :3000
}

caddy run    # in the folder with the Caddyfile

Run a Python app the production way:

pip install gunicorn uvicorn
gunicorn -w 4 -b 127.0.0.1:8000 app:app      # WSGI (Flask/Django), 4 worker processes
uvicorn main:app --workers 4 --port 8000     # ASGI (FastAPI)

A quick load test, then compare a static file with a dynamic endpoint:

hey -n 5000 -c 100 http://localhost:8080/            # static file from Nginx
hey -n 5000 -c 100 http://localhost:8081/api/items   # proxied to your app
ab -n 5000 -c 100 http://localhost:8080/             # the same with ApacheBench

Compare requests per second and the latency percentiles: the static file is usually far faster, and that difference is the cost of running your code. On a real server, watch the logs while the test runs:

sudo tail -f /var/log/nginx/access.log
sudo tail -f /var/log/nginx/error.log
awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn   # status codes (default "combined" format)

#Summary

Key takeaways

• A web server speaks HTTP: accept (and TLS), parse, decide (file, app, redirect, reject), respond, log. It serves static files itself and passes dynamic requests to an application server that runs your code. Reverse proxy and load balancer are roles the same software often also plays.
• Apache: virtual hosts, Directory rules, modules, and .htaccess (convenient for shared hosting, slow because it is read on every request). Its MPM decides concurrency: prefork (process per connection, needed for mod_php), worker (threads), event (threads plus a listener for idle keep-alives).
• Nginx: a root master binds ports and manages unprivileged workers, one per core, each an event loop over thousands of non-blocking connections, so memory stays flat. Config is http → server → location (exact, ^~, regex in order, longest prefix), with try_files, proxy_pass, upstream, proxy_cache, limit_req. nginx -t then nginx -s reload swaps workers without dropping connections.
• Caddy gets and renews certificates automatically through ACME (HTTP-01 needs DNS pointing at it and port 80 open). IIS for Windows and ASP.NET, LiteSpeed as an .htaccess-compatible Apache replacement, HAProxy, Envoy, Traefik, Varnish and cloud load balancers fill the other roles.
• App server interfaces: CGI (process per request) → FastCGI / PHP-FPM (pool, pm.max_children), WSGI (Gunicorn) and ASGI (Uvicorn), Rack (Puma), Servlets (Tomcat, often embedded in Spring Boot), Kestrel; Node.js and Go apps are their own servers.
• Debug by who generated the code: 502 (upstream down/refused/crashed), 504 (upstream too slow), 499 (client gave up), 503 (no healthy backend, maintenance or rate limit), 413 (client_max_body_size), 403 (permissions/index), redirect loops (X-Forwarded-Proto). The error log names the cause; timeouts must nest from outside in.
• In system design: CDN → load balancer → stateless web tier → stateless app tier → cache and database, scaled horizontally, with health checks and graceful deploys.

Self-check: web servers and app servers33 questions