A Stateless App Grows Up

By the end of this lesson you will be able to

• Explain what a stateless application is, what it truly requires (no session in memory, no files on the instance that matter), and how to tell stateless from stateful.
• Walk through the evolution of a simple web app, from one EC2 instance with an Elastic IP to a multi-AZ fleet behind a load balancer, and name the problem that forces each step.
• Explain why vertical scaling means downtime, why many Elastic IPs and many A records do not scale, and how TTL caching keeps a dead address in use.
• Design the standard stateless web tier: Route 53 Alias → ALB with health checks in at least two AZs → private instances in an Auto Scaling group, with security groups chained ALB → instances.
• Cut the cost of that design by committing to the always-on minimum (Reserved Instances or Savings Plans) and using On-Demand or Spot above it.
• Recognise the exam patterns: “scale out to thousands of instances without losing data”, “static IP for an allow-list”, “survive an AZ failure”, “reduce cost of an always-on minimum”.

#How this chapter works

Every service in this chapter has already been taught: EC2, Elastic IPs, security groups, load balancers, Auto Scaling groups, RDS, ElastiCache, Route 53. Chapter 9 is about putting them together the way a solutions architect does:

1. Start with the simplest thing that works.
2. Find the concrete problem it has: downtime, a failure, a bill, a security hole.
3. Apply the smallest fix for that problem, and look for the next one.

The exam asks exactly this: “here is an architecture and a new requirement, what is the best next step?”. Knowing each service is not enough; you need to know which problem each one solves. This lesson follows a stateless app. Lesson 9.2 adds state (a shopping cart and a database), lesson 9.3 adds shared files and fast launches, and lesson 9.4 lets Elastic Beanstalk build the whole stack for you.

Our app is whatisthetime.example, at api.whatisthetime.example. It does one thing: it tells you what time it is. It needs no database. The owners want to start small and can accept some downtime at first, but eventually they want to scale both vertically and horizontally with no downtime at all.

#What “stateless” means

#Every request stands on its own

An application tier is stateless when a server keeps nothing about a client between requests. Each request carries everything needed to answer it, and the answer does not depend on which server receives it or on what that server handled before.

#What stateless really requires

“Our app is stateless” is easy to say and easy to get wrong. A web tier is stateless only if all of these are true:

Local caches are fine as long as losing them costs only speed, never correctness: a cache that can be rebuilt from the database is not state.

#Stateless or stateful? A quick test

Ask one question about the tier: “Can I terminate any instance, at any moment, and send the next request somewhere else, without any user noticing anything but a retry?”

• Yes → the tier is stateless. You can put it behind a load balancer, scale it in and out freely, use Spot Instances, and replace instances during deployments.
• No → something on the instance is state. Find it (sessions, files, local data) and move it out, or the tier will fight every scaling and failure event.

The database itself is of course stateful: state does not disappear, it moves to a tier designed to keep it safe. The point of the design is that the many, cheap, replaceable web instances keep nothing, and the few, carefully protected data stores keep everything.

#The journey, one problem at a time

The animation below is the whole lesson in one place. Pick a stage, press Send requests to see it work, then press the red button to see what breaks and move on to the stage that fixes it. The sections after it explain each stage.

#Stage 1: one instance with an Elastic IP

The simplest version is one small instance, a t2.micro (1 vCPU, 1 GiB of memory), in a public subnet. Users need a fixed address, so you attach an Elastic IP, a static public IPv4 address that stays allocated to your account until you release it (lesson 3.6). An ordinary auto-assigned public IP would change every time the instance is stopped and started; the Elastic IP does not.

It works, it is cheap, and it has two problems that will show up soon: one instance has a fixed amount of capacity, and one instance is a single point of failure. If it, its host or its AZ fails, the site is down.

#Stage 2: scaling vertically

Traffic grows and the t2.micro runs at 100% CPU. The quickest fix is a bigger instance: scale up to an m5.large (2 vCPU, 8 GiB). On an EBS-backed instance this is a stop, change type, start (lesson 6.1):

• Downtime while it is stopped, booting and warming up. Every later resize, up or down, costs another outage.
• The Elastic IP stays attached through the stop and start, so users keep the same address. (Without it, the public IP would change.)
• There is a ceiling: the largest instance type. And however big it is, it is still one instance in one AZ.

Vertical scaling is useful (no code changes, and it is the normal way to grow a relational database primary), but for a web tier it trades one problem for another. The owners wanted to scale with no downtime, so we need more instances, not a bigger one.

#Stage 3: scaling out with an Elastic IP each

Scaling out (horizontally) means running several copies of the app. Because the app is stateless, every copy gives the same answer, so this works immediately. The first attempt: three instances, each with its own Elastic IP.

The trouble is the addresses:

• Users, or their apps, must know all three addresses and choose one. Every new instance means telling every client about a new address; removing one means telling them to stop using it.
• The default quota is 5 Elastic IPs per Region per account (it can be raised on request, but it shows this is not how AWS expects you to scale).
• Since February 2024 every public IPv4 address costs $0.005 per hour, Elastic or auto-assigned, attached or idle: about $3.65 a month each.
• If an instance dies, the users using its address simply fail. An Elastic IP points at one instance and knows nothing about health. You can remap it to a spare instance (lesson 3.6), but that is a manual, one-at-a-time fix.

We need one name for the whole fleet that never changes for the client.

#Stage 4: Route 53 A records with several values

So we put DNS in front and drop the Elastic IPs. The instances keep ordinary public IPs, and a simple A record for api.whatisthetime.example lists all of them, with a TTL of 1 hour (3,600 seconds). Route 53 returns all values in random order and each client uses one (lesson 8.3). Clients now know one name, and adding an instance means adding a value to the record.

This is better, but DNS answers are cached (lesson 8.1), and simple routing has no health checks:

• An instance dies, its address lives on. Every resolver and client that cached the answer keeps sending requests to the dead address until the TTL runs out, up to an hour later. Even new answers include it until someone edits the record.
• A new instance waits. Clients that cached the old answer do not see the new address until their cache expires.
• A short TTL (say 60 seconds) shrinks the window but multiplies DNS queries, and DNS still cannot see health or react in seconds. Multi-value answer routing can leave out unhealthy addresses, but cached copies still go stale, and it is still not a load balancer (lesson 8.3).

#Stage 5: a load balancer with health checks

The fix is to put something in the traffic path that knows which instances are healthy: an Application Load Balancer (lessons 6.2 and 6.3).

• Route 53 gets an Alias record for api.whatisthetime.example pointing at the ALB (lesson 8.2). The ALB has a DNS name but no fixed IP addresses (they change as it scales), so you never type its IPs into an A record. An Alias works at the zone apex too, and Alias queries to a load balancer are free.
• Every request reaches the ALB, which sends it to a healthy target. Its health checks find a dead instance within seconds and stop routing to it. Clients cache only the ALB’s name and addresses, never an instance address, so there is nothing stale to hit.
• The instances no longer need to be reachable from the internet. Move them to private subnets without public IPs; the ALB, in public subnets, is the only public entry point. That also removes their public IPv4 charges.
• Chain the security groups (lesson 3.5): the ALB’s security group allows HTTP/HTTPS from 0.0.0.0/0; the instances’ security group allows the app port only from the ALB’s security group, as a source, not from an IP range. Nobody can reach an instance except through the load balancer.

Now losing an instance is a non-event. What remains is that someone still has to add instances when traffic rises, remove them when it falls, and replace them when they die.

#Stage 6: an Auto Scaling group

An Auto Scaling group (lesson 6.5) does that work:

• It launches instances from a launch template (the AMI and user data from lesson 4.3) and registers them with the ALB’s target group automatically.
• A scaling policy (for example target tracking on average CPU or on requests per target) moves the desired capacity between min and max as the load changes. At night it scales in to the minimum, and terminating instances costs nothing because they hold no state.
• With the ELB health check type turned on, an instance that fails the ALB’s health check is terminated and replaced. With only the default EC2 status checks, the ASG would replace crashed instances but keep a running instance whose app has hung.

The last weakness is location. If every instance sits in one AZ, one data-centre event takes them all out together.

#Stage 7: multi-AZ

Spread everything across at least two Availability Zones:

• The ALB must be enabled in at least two AZs: you cannot create an ALB with fewer than two AZ subnets. It runs a node in each, and cross-zone load balancing (always on for an ALB) spreads requests evenly over all healthy targets.
• The ASG gets subnets in two or three AZs and keeps the instances balanced across them. If an AZ fails, the ALB stops sending traffic there, and the ASG launches replacements in the healthy AZs (the default AZ distribution strategy, balanced best effort, moves launches to a healthy AZ when one is impaired).
• Size for the loss of an AZ. If the peak needs 6 instances, two AZs need 6 each to survive losing one (12 in total), three AZs need 3 each (9). This is the N+1 rule from lesson 6.1; without it, the survivors are overloaded until new instances are ready.

#Stage 8: reserve the minimum

The design is now available and scalable, and the remaining issue is the bill. With a minimum of 2 (one per AZ), two instances run 24 hours a day, 365 days a year, whatever the traffic. Paying On-Demand rates for something that never switches off is the most expensive way to buy it.

• Cover the always-on minimum with Reserved Instances or a Savings Plan for 1 or 3 years (lesson 4.4): up to about 72% off. It is used every hour, so not one committed hour is wasted.
• Pay On-Demand for the instances above the minimum, or run them on Spot (up to about 90% off) with a mixed instances policy: a stateless instance that is reclaimed with two minutes’ notice loses nothing.
• Do not reduce the minimum to 1 or 0 to save money: that removes high availability, which was a requirement.

The practice question in lesson 6.5 (an ASG with a minimum of 2 behind an ALB; “reserve two EC2 instances” is the answer) is exactly this stage. Here is the same idea as a calculator. Try covering 2, then 6, and switch the extra instances to Spot:

Above the floor, a commitment pays only for the hours it is actually used, so reserving the peak is a bet that the traffic curve will stay the same for years. The floor is the one part you know for certain.

#What each stage fixed and what it broke

#The well-architected view

AWS describes good architectures with the pillars of the Well-Architected Framework. This short story touched five of them:

• Reliability: health checks, an Auto Scaling group that replaces instances, and two or more AZs. No single instance or AZ can take the site down.
• Performance efficiency: scale out instead of up, with capacity that follows the load minute by minute.
• Cost optimisation: scale in at night, cover the always-on minimum with a commitment, use Spot above it, and drop public IPv4 addresses the instances do not need.
• Security: private instances, the load balancer as the only public entry point, security groups that reference the ALB’s security group.
• Operational excellence: no hand-built or hand-replaced instances; everything is launched from a template and replaced automatically.

#What goes wrong

• Hidden state. The app passes a quick test with one instance, then users get logged out or lose uploads once there are several, or after a scale-in. Look for sessions in memory and files on local disk before you scale out.
• Sticky sessions used to hide state. Stickiness (lesson 6.4) sends a user back to the same instance, so the problem seems to go away, until that instance is terminated and its sessions with it. It also skews load. Stickiness is a workaround, not stateless design.
• Instances still public. The ALB was added, but the instances kept public IPs and a security group open to 0.0.0.0/0. Anyone can bypass the load balancer, and you pay for every public IPv4 address.
• ASG on EC2 health checks only. The ALB marks an instance unhealthy, but the group keeps it because its EC2 status checks pass. Turn on the ELB health check type (and a sensible grace period).
• All targets in one AZ, even though the ALB is in two (see the warning above).
• Under-sized AZs. Two AZs with exactly the peak capacity between them: after an AZ failure, half the capacity serves all the traffic until the ASG catches up.
• An A record with the ALB’s IP addresses. They change; use an Alias record (or a CNAME for a non-apex name).
• A minimum of 1 “to save money”. One instance is a single point of failure; reserve the minimum instead of lowering it.

#How it is used in system design

• This is the default shape of a web or API tier on AWS, and the base of almost every scenario in this chapter: DNS name → load balancer → stateless compute in several AZs → state in managed services.
• The same rule holds for containers and serverless: ECS tasks, EKS pods and Lambda functions are also expected to be stateless and disposable. What changes is who manages the fleet.
• Need a static IP for a partner’s allow-list? An ALB has none. Use a Network Load Balancer, which has one static IP per AZ and can take an Elastic IP in each, or put AWS Global Accelerator (a later chapter) in front of the ALB for two static anycast IPs.
• Need it global? Repeat the whole stack in another Region and choose between them with Route 53 latency or failover records (lesson 8.3).
• Statelessness is what makes deployments safe: new instances with the new version can be added and old ones removed at any time (lesson 9.4 shows Beanstalk’s rolling and immutable deployments).

#Try it yourself

The commands below build stages 1, 2 and 4–8 by hand with the AWS CLI. IDs are made up; replace them with yours.

# Stage 1: allocate an Elastic IP and attach it to the instance
aws ec2 allocate-address --domain vpc
aws ec2 associate-address --instance-id i-0abc1234567890def --allocation-id eipalloc-0abc1234567890def

# Stage 2: scale up (stop, change type, start). The Elastic IP stays attached.
aws ec2 stop-instances --instance-ids i-0abc1234567890def
aws ec2 wait instance-stopped --instance-ids i-0abc1234567890def
aws ec2 modify-instance-attribute --instance-id i-0abc1234567890def --instance-type '{"Value": "m5.large"}'
aws ec2 start-instances --instance-ids i-0abc1234567890def

# Stage 4: one simple A record with several values and a 1-hour TTL
aws route53 change-resource-record-sets --hosted-zone-id Z0EXAMPLE12345 --change-batch '{
  "Changes": [{ "Action": "UPSERT", "ResourceRecordSet": {
    "Name": "api.whatisthetime.example", "Type": "A", "TTL": 3600,
    "ResourceRecords": [{"Value": "198.51.100.21"}, {"Value": "198.51.100.22"}, {"Value": "198.51.100.23"}] } }] }'

# Stage 5: security groups chained ALB -> instances
aws ec2 create-security-group --group-name alb-sg --description "ALB" --vpc-id vpc-0abc1234
aws ec2 authorize-security-group-ingress --group-id sg-0a11b22c33d44e55f --protocol tcp --port 443 --cidr 0.0.0.0/0
aws ec2 create-security-group --group-name web-sg --description "Web instances" --vpc-id vpc-0abc1234
aws ec2 authorize-security-group-ingress --group-id sg-0f55e44d33c22b11a --protocol tcp --port 80 \
  --source-group sg-0a11b22c33d44e55f

# Stage 5 and 7: an ALB needs subnets in at least two AZs
aws elbv2 create-load-balancer --name time-alb --type application \
  --subnets subnet-0aaa1111 subnet-0bbb2222 --security-groups sg-0a11b22c33d44e55f
aws elbv2 create-target-group --name time-tg --protocol HTTP --port 80 --vpc-id vpc-0abc1234 \
  --health-check-path /health --health-check-interval-seconds 10 --unhealthy-threshold-count 2

# Stage 6 and 7: an ASG across two private subnets in two AZs, using ELB health checks
aws autoscaling create-auto-scaling-group --auto-scaling-group-name time-asg \
  --launch-template LaunchTemplateName=time-web,Version='$Latest' \
  --min-size 2 --max-size 8 --desired-capacity 2 \
  --vpc-zone-identifier "subnet-0ccc3333,subnet-0ddd4444" \
  --target-group-arns arn:aws:elasticloadbalancing:us-east-1:111122223333:targetgroup/time-tg/0123456789abcdef \
  --health-check-type ELB --health-check-grace-period 120

# Stage 5: the Alias record to the ALB (take the zone ID and DNS name from describe-load-balancers)
aws elbv2 describe-load-balancers --names time-alb \
  --query "LoadBalancers[0].[DNSName,CanonicalHostedZoneId]"

# Stage 8: what would a Savings Plan or RI for the always-on part save?
aws ce get-savings-plans-purchase-recommendation --savings-plans-type COMPUTE_SP \
  --term-in-years ONE_YEAR --payment-option NO_UPFRONT --lookback-period-in-days THIRTY_DAYS

Then watch the behaviour from a client:

dig +noall +answer api.whatisthetime.example     # TTL counting down in the second column
for i in 1 2 3 4 5; do curl -s https://api.whatisthetime.example/; done   # any instance answers

#Summary

Key takeaways

• Stateless = the server keeps nothing about a client between requests, so any instance can answer any request and any instance can be lost. It requires: no session in memory, no files that matter on local disk, no data only on one instance. State moves to shared stores (ElastiCache, DynamoDB, RDS, S3, EFS).
• Exam trick: “scale horizontally to thousands of instances without losing data” → make the app stateless.
• Vertical scaling (t2.micro → m5.large) needs a stop and start on EBS-backed instances: downtime, a ceiling, still a single point of failure. An Elastic IP survives the stop.
• One Elastic IP per instance does not scale: clients must know every IP, the default quota is 5 per Region, every public IPv4 costs $0.005 per hour, and there is no failover.
• Several A record values give one name, but clients cache answers for the TTL (a dead IP can be used for up to an hour with TTL 3,600), and simple routing has no health checks.
• ALB + health checks + Route 53 Alias: failover within seconds, no instance IPs cached, Alias because the ALB’s IPs change. Instances go private, and their security group allows only the ALB’s security group.
• ASG with ELB health checks: scale out, scale in and replace automatically.
• Multi-AZ: the ALB needs at least two AZs; give the ASG subnets in two or more AZs and size each AZ so the others can carry the peak.
• Cost: reserve the always-on minimum (RI or Savings Plan), On-Demand or Spot above it; never lower min below what HA needs.

Self-check: a stateless app grows up31 questions