Load Balancing, Health Checks and TCP vs UDP

By the end of this lesson you will be able to

• Explain what a load balancer does and list the jobs it takes off your servers: one DNS name, spreading load, health checks, TLS offload, stickiness, multi-AZ availability and separating public from private traffic.
• Say why a managed Elastic Load Balancer usually beats running your own, and name the services it plugs into.
• Tell the four ELB types apart (CLB, ALB, NLB, GWLB) by layer and protocol, and know which one is the previous generation.
• Configure health checks (protocol, port, path, interval, timeout, thresholds, success codes) with today’s ALB and NLB defaults, and calculate how long a failure takes to detect.
• Read the target health states, explain fail-open, and troubleshoot an instance that is “running but unhealthy”.
• Lock instances down so they accept traffic only from the load balancer’s security group, on ALB and on NLB.
• Explain why clients must use the load balancer’s DNS name, and why TCP vs UDP decides which load balancer you can use.

#What a load balancer is

Chapter 6.1 ended with an application running on several EC2 instances in at least two Availability Zones. That raises an obvious question: which address do users connect to? Give them the IP of one instance and that instance takes all the traffic and becomes a single point of failure. Give them a list of IPs and every client has to pick one, notice when it dies and try another.

A load balancer solves this. It is a server (in AWS, a fleet of servers you never see) that sits in front of your instances, accepts every client connection, and forwards each request to one of several servers behind it, called targets. Clients only ever talk to the load balancer.

#Why use a load balancer

#Why a managed load balancer: Elastic Load Balancing

You could install HAProxy or Nginx on an EC2 instance and run your own load balancer. Then you also own its problems: it is now your single point of failure, so you need two of them in two AZs, a way to fail over between them, patching, scaling when traffic grows, certificates, logs and monitoring.

Elastic Load Balancing (ELB) is the managed version:

• AWS guarantees it works: it runs across the AZs you choose, replaces failed nodes and scales its own capacity as traffic changes.
• AWS does the upgrades, maintenance and high availability. You get a small set of settings (listeners, rules, target groups, health checks) rather than a whole server to look after.
• It is integrated with the rest of AWS (next section).
• Cost: on the bill, a small self-managed proxy on one instance can look cheaper than an ELB (an ALB starts at roughly $16 a month in us-east-1 before usage-based capacity charges). But once you add a second instance for availability and count the hours spent running it, the managed service is almost always the cheaper choice overall. Exam questions that say “least operational overhead” point to ELB.

#The ELB family

AWS offers four kinds of managed load balancer:

The layer is the key idea. An ALB opens each HTTP request and can read its path, headers and host name. An NLB never looks inside: it forwards connections and datagrams by IP address and port, which is why it is so fast. A GWLB passes whole IP packets to a fleet of appliances. Lesson 6.3 covers each one in depth; for now, remember the layer and the protocols.

• Internet-facing or internal: every type except GWLB can be created internet-facing (public DNS name, nodes with public IPs, in public subnets) or internal (private IPs only, reachable from inside the VPC and connected networks). A classic three-tier design uses both: an internet-facing ALB for the web tier and an internal one in front of the application tier.
• AZs: an ALB must be placed in subnets in at least two AZs. An NLB can use one, but production should still use two or more.

#What ELB integrates with

A load balancer does not work alone; it receives traffic and hands it to other services, and it uses others for certificates, names and protection.

#Health checks

A load balancer does not send traffic blindly. It keeps asking every target “are you able to serve?” and only routes to targets that keep saying yes. Health checks are configured on the target group, the set of targets a listener forwards to (lesson 6.3 covers target groups in detail).

#What is checked

A health check is a small request the load balancer sends to each target on a schedule:

• Protocol: HTTP or HTTPS for an ALB (an HTTP GET); TCP, HTTP or HTTPS for an NLB. A TCP check only proves the port accepts a connection; an HTTP check proves the application answers.
• Port: by default the traffic port, the same port the target receives requests on. You can choose another, for example a separate admin port such as 8080.
• Path: for HTTP/HTTPS checks, the URL to request. The default is /, but a dedicated endpoint such as /health is better: cheap, no login, and it can check the things the app really needs (database connection, disk space).
• Success codes: which HTTP status codes count as healthy. Anything else, or no answer within the timeout, is a failure.

#Timing and thresholds

Two numbers matter most:

• Unhealthy threshold × interval ≈ how long a broken target keeps getting traffic. With ALB defaults, 2 × 30 s = about 60 seconds (plus the 5-second timeout if the target hangs instead of returning an error).
• Healthy threshold × interval ≈ how long a repaired target waits before it gets traffic again: 5 × 30 s = 150 seconds.

Coming back is deliberately slower than leaving, so a server that keeps crashing and restarting does not flap in and out of service. One exception: when a target is first registered, an ALB needs it to pass one check to become healthy.

#Worked example: tuning for faster failover

A checkout service gets 50 requests per second per instance. When an instance hangs, how many requests fail before the ALB stops using it?

• Defaults (interval 30 s, timeout 5 s, unhealthy threshold 2): worst case 2 × 30 + 5 = 65 s, so about 50 × 65 = 3,250 failed requests.
• Interval 10 s, timeout 5 s, unhealthy threshold 2: worst case 2 × 10 + 5 = 25 s, so about 1,250.

Faster checks find problems sooner but cost a little more load on the targets and make a single slow response more likely to remove a healthy instance. Most teams lower the interval to 10–15 seconds for latency-sensitive services and keep the defaults elsewhere. Clients should also retry idempotent requests, because some requests always land during the detection window.

#Target health states

The Targets tab of a target group (EC2 console → Load Balancing → Target Groups) shows each target’s state:

An NLB adds one more, unhealthy.draining: an unhealthy target that keeps its existing connections for a grace period but gets no new ones.

#When every target is unhealthy: fail open

If all targets in a target group are unhealthy at the same time (in every enabled AZ), the load balancer fails open: it routes requests to all of them anyway, as if they were healthy. The logic is that sending traffic nowhere guarantees an outage, while the health check itself may be what is broken (a renamed /health path, a dependency in the check that is down, a security group change). An NLB also fails open when the target group is empty.

So “all targets unhealthy but users are still being served” is not a bug; it is fail-open, and it is a strong hint that the health check configuration is wrong rather than the app. For an NLB, when a single AZ has no healthy targets, that AZ’s load balancer IP is removed from DNS so clients stop going there.

#Troubleshooting “running but unhealthy”

#Security groups: trust only the load balancer

The safest design has no path from the internet to an instance except through the load balancer. You build it with two security groups (lesson 3.5):

Why a security group reference and not an IP range?

• The load balancer’s nodes change IP addresses as it scales and replaces nodes. A rule that names sg-alb matches any interface that carries that group, whatever its IP.
• A rule with 0.0.0.0/0 on the instances lets anyone who finds an instance’s address (a public IP left on by mistake, or a peered network) bypass the load balancer, its TLS, its WAF rules and its logs.
• A rule with the VPC CIDR is better than 0.0.0.0/0 but still lets every other machine in the VPC in.

#Network Load Balancers and security groups

For years an NLB had no security group at all. It passed the client’s IP through to the target, so the instance’s security group had to allow the clients’ IP ranges (often 0.0.0.0/0) plus the NLB’s own private IPs for health checks. There was no way to say “only via the NLB”.

Since August 2023 an NLB can have security groups, and the same chaining works: the NLB’s group filters clients, and the targets reference the NLB’s group. Points to remember:

• You must attach a security group when you create the NLB. An NLB created without one can never get one later; you would have to create a new NLB. Once it has at least one, you can change them freely.
• Health checks are subject to the NLB’s outbound rules, not its inbound rules. Make sure the outbound rules allow the health check port to the targets.
• The targets must allow both the traffic port and the health check port from the NLB’s group, even when client IP preservation is on.
• For traffic that arrives through interface VPC endpoints, you can choose whether the NLB’s inbound rules apply.
• A GWLB has no security groups; filtering there is the appliances’ job.

#Reached by DNS name, not by IP

Every load balancer gets a DNS name such as my-alb-1234567890.us-east-1.elb.amazonaws.com. Behind that name are the load balancer’s nodes, at least one per enabled AZ. For an ALB (and a CLB) the IP addresses behind the name change as AWS scales the load balancer up and down and replaces nodes; the records have a short TTL of 60 seconds so clients follow those changes quickly.

So:

• Always connect by the DNS name, never by an IP you looked up once.
• Point your own domain at it with a Route 53 alias record (a CNAME also works for subdomains, but not for the zone apex). Chapter 8 covers alias records.
• When a customer insists on fixed IPs to put in a firewall allow list, the answers are an NLB with one Elastic IP per AZ or Global Accelerator in front of an ALB. Lesson 6.3 covers both.

#TCP vs UDP: a quick refresher

To choose a load balancer you need to know what kind of traffic you have. Almost everything on the internet travels over one of two transport protocols. The System Design course covers them in depth in TCP, UDP and TLS; here is what you need for this chapter.

Why the uses fall that way:

• Web browsing: a missing image is annoying, but one missing line of code breaks the page, so the web runs on TCP (HTTP/3 is the exception, see below).
• Databases: a bank transfer cannot lose a single byte. TCP.
• File transfer and email: a word missing from the middle of a file or an email is not acceptable. TCP.
• Live video calls (Zoom, Meet): losing a frame or two makes the picture stutter for a moment, which you ignore. Freezing the call for seconds while lost frames are resent, as TCP would, is far worse. UDP.
• VoIP: a tiny gap in the sound is fine; delay makes conversation impossible. UDP.
• Online multiplayer games: player positions must arrive within milliseconds; an old position is useless anyway. UDP.
• DNS: one small question and one small answer; a handshake would double the time. UDP on port 53 (TCP for large answers and zone transfers).

#Why the load balancer cares

• An ALB only understands HTTP and HTTPS (which run on TCP). It has no UDP listener, so it cannot carry a game’s UDP traffic, VoIP, or DNS.
• An NLB works at layer 4: it supports TCP, UDP and TLS listeners (and TCP_UDP on the same port). It does not open the payload; it looks only at IPs and ports, so it handles millions of requests per second at very low latency. Since November 2025 it can also pass QUIC traffic through, keeping a session on the same target by its connection ID even when a phone changes networks (a new feature; current exam questions still focus on TCP and UDP).
• UDP targets are health-checked with TCP, HTTP or HTTPS, because there is no generic way to ask a UDP service “are you alive?”. Run a small check endpoint that reports whether the UDP service is working.

#How load balancers are used in system design

• Every web tier: an internet-facing ALB in public subnets of two or three AZs, instances or containers in private subnets, the instances’ security group referencing the ALB’s. This is the default shape of almost every AWS architecture diagram.
• Internal tiers: an internal load balancer between the web tier and the application tier, so each tier scales and fails independently.
• Load balancer + Auto Scaling group: the ASG adds and removes instances; the load balancer registers them and stops using failed ones; the ASG uses the ELB health check to replace instances that are running but broken (lesson 6.5).
• Deployments: deregister a few targets, update them, register them back; health checks make sure only working versions take traffic. Blue/green deployments switch traffic between two target groups.
• Non-HTTP services: game servers, message brokers, VoIP gateways and databases sit behind an NLB.
• What a load balancer does not fix: if the application keeps session data in memory, spreading requests breaks logins and carts. Stickiness is a quick patch (lesson 6.4); storing the session outside the instance is the real fix (chapter 9).

#Try it yourself

# 1. Two security groups: the ALB open to the internet, the instances open only to the ALB
SG_ALB=$(aws ec2 create-security-group --group-name sg-alb --description "ALB" \
  --vpc-id $VPC --query GroupId --output text)
aws ec2 authorize-security-group-ingress --group-id $SG_ALB --protocol tcp --port 443 --cidr 0.0.0.0/0
aws ec2 authorize-security-group-ingress --group-id $SG_ALB --protocol tcp --port 80  --cidr 0.0.0.0/0

SG_APP=$(aws ec2 create-security-group --group-name sg-app --description "web instances" \
  --vpc-id $VPC --query GroupId --output text)
aws ec2 authorize-security-group-ingress --group-id $SG_APP --protocol tcp --port 80 \
  --source-group $SG_ALB              # a security group reference, not a CIDR

# 2. A target group with an explicit health check
TG=$(aws elbv2 create-target-group --name web-tg --protocol HTTP --port 80 --vpc-id $VPC \
  --health-check-protocol HTTP --health-check-path /health \
  --health-check-interval-seconds 10 --health-check-timeout-seconds 5 \
  --healthy-threshold-count 3 --unhealthy-threshold-count 2 \
  --matcher HttpCode=200 \
  --query "TargetGroups[0].TargetGroupArn" --output text)

# 3. An internet-facing ALB in public subnets of two AZs, and a listener
ALB=$(aws elbv2 create-load-balancer --name web-alb --type application --scheme internet-facing \
  --subnets $PUB_A $PUB_B --security-groups $SG_ALB \
  --query "LoadBalancers[0].LoadBalancerArn" --output text)
aws elbv2 create-listener --load-balancer-arn $ALB --protocol HTTP --port 80 \
  --default-actions Type=forward,TargetGroupArn=$TG

# 4. Register two instances (launched in private subnets with sg-app) and watch their health
aws elbv2 register-targets --target-group-arn $TG --targets Id=$I1 Id=$I2
aws elbv2 describe-target-health --target-group-arn $TG \
  --query "TargetHealthDescriptions[].[Target.Id,TargetHealth.State,TargetHealth.Reason]" --output table

# 5. Use the DNS name (never the IPs behind it)
DNS=$(aws elbv2 describe-load-balancers --load-balancer-arns $ALB \
  --query "LoadBalancers[0].DNSName" --output text)
dig +short $DNS                       # the IPs here can change over time
curl -s http://$DNS/                  # repeat: responses come from different instances

# 6. Break one target and watch it go unhealthy (on that instance)
sudo systemctl stop httpd             # about 2 × 10 s later describe-target-health shows unhealthy
sudo systemctl start httpd            # about 3 × 10 s later it is healthy again

# 7. An NLB with a security group (must be chosen at creation) and a UDP listener
NLB=$(aws elbv2 create-load-balancer --name game-nlb --type network --scheme internet-facing \
  --subnets $PUB_A $PUB_B --security-groups $SG_NLB \
  --query "LoadBalancers[0].LoadBalancerArn" --output text)
aws elbv2 create-target-group --name game-udp --protocol UDP --port 7777 --vpc-id $VPC \
  --health-check-protocol TCP --health-check-port 8080      # UDP targets get a TCP/HTTP check

#Summary

Key takeaways

• A load balancer gives clients one DNS name, spreads requests over many targets, health-checks them, terminates TLS, can keep users sticky, runs across AZs, and keeps instances private.
• ELB is managed: AWS runs, scales, patches and keeps it available. Fewer knobs, far less work than your own proxy fleet. It integrates with EC2, ASG, ECS, Lambda, ACM, CloudWatch, Route 53, WAF and Global Accelerator.
• Types: CLB (previous generation, migrate away), ALB (layer 7, HTTP/HTTPS/gRPC/WebSocket), NLB (layer 4, TCP/UDP/TLS, static IPs, extreme performance), GWLB (layer 3, inline appliances).
• WAF attaches to the ALB (and CloudFront, API Gateway, AppSync, Cognito, App Runner, Verified Access, Amplify), not to the CLB or NLB.
• Health checks: protocol, port, path, success codes. Defaults: interval 30 s, unhealthy threshold 2, healthy threshold 5; timeout 5 s (ALB) or 10 s / 6 s (NLB TCP-HTTPS / HTTP); success codes 200 (ALB) or 200–399 (NLB).
• Detection ≈ unhealthy threshold × interval (≈ 60 s by default); recovery ≈ healthy threshold × interval (≈ 150 s).
• States: initial, healthy, unhealthy, unused, draining, unavailable (NLB also unhealthy.draining). If all targets are unhealthy, the load balancer fails open.
• “Running but unhealthy” → app crashed, security group blocks the check port, wrong path or redirect, or the app is slow to start.
• Instances allow the app port only from the load balancer’s security group. NLBs support security groups since 2023, but only if attached at creation.
• Use the DNS name: an ALB’s IPs change. Fixed IPs → NLB with Elastic IPs or Global Accelerator.
• TCP = handshake, reliable, ordered (web, databases, files, email). UDP = no handshake, no guarantees, fast (DNS, VoIP, games, live media). ALB cannot do UDP; NLB can.

Self-check: load balancing basics30 questions