A Stateful Shop: Sessions, Data and Read Scaling

By the end of this lesson you will be able to

• Explain what makes a web application stateful, and why a shopping cart breaks when requests are spread over several servers.
• Walk a design from “cart in server memory” through sticky sessions, a cart in a cookie and server-side sessions in ElastiCache or DynamoDB, and name the cost of each step.
• Decide where each kind of data belongs: short-lived session state, durable user data in RDS, and hot read-only data in a cache.
• Choose between read replicas and ElastiCache for read scaling, and predict what replica lag and stale cache entries do to users.
• Make every data tier survive an AZ failure with Multi-AZ, and lock the tiers together with security groups that reference each other.
• Recognise the exam wording for each step: “session lost when an instance terminates”, “legacy app, no code change”, “scale reads without code changes”, “restrict access to the database”.

#A shop that has to remember

#The My Cloth story and its requirements

In lesson 9.1, the stateless app only had to tell the time. Any server could answer any request, because no answer depended on an earlier one. Now meet My Cloth, an online clothes shop. It has a shopping cart, hundreds of shoppers at the same moment, and these requirements:

• scale horizontally (more servers, not a bigger one) and keep the web tier as stateless as possible;
• shoppers must never lose their cart, even when a server fails or Auto Scaling removes one;
• shopper details (name, address, order history) must be kept in a database;
• the whole thing must survive the loss of an AZ and be locked down so that only the right tier can reach the data.

We start from the end of lesson 9.1: an ALB in front of an Auto Scaling group spread over several AZs. Then we hit one problem at a time and fix it, the way a solutions architect would.

#Stateless vs stateful

A server is stateless when it needs nothing from earlier requests to answer the current one. It is stateful when it must remember something between requests, the state: who is logged in, what is in the cart, which step of a checkout the shopper has reached.

HTTP itself is stateless; the state has to live somewhere: in a server’s memory, in the client, or in a shared store. That choice decides how well the application scales and what happens when a server dies. The System Design lesson State, cookies and sessions explains cookies, sessions and tokens in general; this lesson applies them to AWS.

#Stage 1: the cart lives in server memory

The first version of the shop keeps each cart in the RAM (or local disk) of the server that handled the request. With one server that works. With several servers behind a load balancer it falls apart.

The load balancer is doing exactly its job: it balances every request on its own (lesson 6.4). The design is wrong, not the ALB. Watch the problem, and the next four stages, in one diagram:

#Stage 2: sticky sessions

#The quick fix

The fastest fix needs no code: turn on stickiness (session affinity) on the ALB’s target group. The ALB gives the browser a cookie (AWSALB for duration-based stickiness, or AWSALBAPP that follows your own application cookie), and every later request with that cookie goes to the same target. The shopper keeps landing on server 1, where the cart is.

#What lesson 6.4 already showed

Lesson 6.4 took this apart in detail; here is the short version. The session ID and the session data are two different things. The cookie is only a token, like a restaurant token that says “counter 1”; the cart itself is written in server 1’s RAM, like the order on the cook’s pad. When server 1 crashes or restarts, the RAM is wiped. The next request still carries the cookie, the ALB sees that server 1 is gone and is forced to pick server 2, and server 2 answers “Your cart is empty”. With stickiness the data is trapped inside one server: when the server dies, the data dies with it.

#The costs of stickiness

• Lost carts on instance death. A crash, a failed health check or a hardware retirement takes every session on that instance with it.
• Lost carts on scale-in and deployments. Auto Scaling chooses which instance to terminate by its termination policy, not by who is shopping on it (lesson 6.5). Deregistration delay lets open requests finish, but the RAM still goes. A rolling deployment replaces every server, so it can empty every cart.
• Uneven load. Shoppers stay on the server they were stuck to. Busy servers stay busy, and an instance launched by a scale-out receives only new shoppers, so the scale-out helps less than the numbers suggest.
• Cookie-dependent. A client that refuses cookies is routed fresh on every request.

With four servers and an even spread, terminating one instance empties about a quarter of all carts at once.

#Stage 3: the cart in a cookie

#Let the browser carry the state

If the servers must not remember the cart, someone else has to. The simplest “someone else” is the browser. The server writes the whole cart into a cookie, for example cart=shirt-M-blue,trousers-32, and the browser sends it back with every request. Any server can read it, so:

• the web tier is truly stateless: the ALB balances freely, Auto Scaling may terminate any instance, no stickiness is needed;
• there is nothing to store on the server side and nothing extra to pay for.

#Why it does not last

• Size. Browsers are required to store at least 4,096 bytes per cookie (name, value and attributes together, RFC 6265), and in practice that is also about the most they keep; a larger cookie is silently dropped. A cart of twenty items with sizes, colours and prices already reaches it. (The ALB is not the limit: it accepts a 16 KB header and 64 KB of request headers in total.)
• Heavier requests. The browser sends every cookie of the domain with every request, including images, CSS and API calls. A 3.5 KB cookie on a page that makes 40 requests adds about 140 KB of upload per page view, much of it over slow mobile uplinks.
• The client controls it. A shopper can open the developer tools and change the price of the shirt to ৳1. The server must validate every cookie: sign it with an HMAC using a secret only the servers know, and reject any cookie whose signature does not match. A signed cookie can still be read, so it must never contain anything secret; encrypt it if the content is sensitive.
• Recalculation and devices. The server must still check prices and stock against the database at checkout, and the cart does not follow the shopper to another device or browser.

A signed token such as a JWT is this same idea used for identity: small, signed, readable, sent with every request. It works well for “who is this user”, and badly for a growing shopping cart.

#Stage 4: server-side sessions in a shared store

#Only an ID travels

Now combine the good half of each earlier stage. The cookie carries only a session ID: a long random value (for example 128 random bits) that means nothing by itself. The session data, the cart, lives in a shared store outside the web servers, keyed by that ID.

1. First request: create the session

   web-1 creates a random ID, writes the cart to the store under sess:<id> with a TTL of, say, 30 minutes, and returns Set-Cookie: session_id=<id>; HttpOnly; Secure; SameSite=Lax.

2. Any later request: read it from the store

   The next click may land on web-2. It reads sess:<id> from the store, sees the shirt and the trousers, adds the new item, writes the session back and resets the TTL, so an active shopper never expires.

3. A server dies: nothing happens to the cart

   A crash, a scale-in or a deployment removes servers, but no cart was in them. The web tier is stateless again, and stickiness is no longer needed.

This fixes every problem of the cookie cart too: the cookie is tiny, the request stays light, the cart can be as big as the store allows, and a shopper who edits the ID only gets “no such session”: a random 128-bit ID cannot be guessed, so editing it does not open anyone else’s cart.

#ElastiCache or DynamoDB?

Both appear in the notes and the slides as the answer, and both are right. They differ in how they keep the data.

Two details matter for the exam and for production:

• Memcached is a poor session store. It has no replicas and no failover, so a lost node loses its sessions. When the notes say “ElastiCache (Redis/Memcached)”, read it as Valkey or Redis OSS with a replica for sessions (lesson 7.5); Memcached is for a plain cache you can afford to lose.
• DynamoDB TTL is not a timer. It deletes expired items for free, without using write capacity, but only some time after they expire. The application must treat an item whose expires_at is in the past as gone.

Both options need a code change: the application must read and write the session in the store. That is the price of the architectural fix, and the reason stickiness remains the answer only when the code cannot change.

#Try the four designs and break them

Choose a design, add items, then crash a server, scale in, edit the cookie, overflow 4 KB or fail the cache node. Each press is one request:

#Stage 5: user data belongs in a database

A cart can expire after half an hour; a shopper’s name, address, password hash and order history must not. That is durable, relational data, so it goes into Amazon RDS (lesson 7.1), or Aurora (lesson 7.3).

• On login, the web tier reads the account from RDS; at checkout it writes the order in a transaction (stock check, order rows, payment status together), then deletes the cart session.
• The split is now clear: short-lived state (sessions, carts) in ElastiCache or DynamoDB, the system of record in RDS, nothing in the web servers.
• A session can also cache a few account fields (the shopper’s name for the page header) so that not every page view needs RDS.

The database is now the busiest part of the design, and the next two stages are about it: too many reads, and what happens when its AZ fails.

#Stage 6: scaling reads

#Too many reads on one primary

A clothes shop reads far more than it writes: every product page, category list and search hits the database, while only a few requests place orders. At sale time the single RDS instance runs out of CPU. A bigger instance class (vertical scaling) buys time, but has a ceiling. Take the repeated reads off the primary instead, in one of two ways.

#Option 1: read replicas

Create RDS read replicas (up to 15 per source for MySQL, MariaDB, PostgreSQL, Oracle and SQL Server; lesson 7.1). The primary copies every change to them asynchronously, and each replica has its own endpoint. The application sends writes to the primary endpoint and reads to the replicas.

• The change to the application is small: a second connection string for reads. The queries stay the same SQL.
• Replicas answer any query, so they suit general browsing, search and reporting.
• The catch is replica lag. A replica can be a moment behind: the last jacket is sold on the primary, and for a second or two the replica still shows “1 left”. Use replicas for reads that may be slightly old, and send decisions (stock at checkout, the order total) and a user’s own fresh writes (read-your-own-writes) to the primary.

#Option 2: ElastiCache in front of RDS (lazy loading)

Put ElastiCache in front of the database and use lazy loading (cache-aside, lesson 7.5):

1. Read the cache first, for example product:42.
2. Hit: answer from memory in well under a millisecond; RDS does nothing.
3. Miss: read RDS, answer, and write the result into the cache with a TTL, so the next read hits.

With a 95% hit ratio, 10,000 product reads per second become 500 reads per second on RDS. The catch has two parts: the application must be written to use the cache (a real code change), and a cached value can be stale: a price changed in RDS stays old in the cache until its TTL runs out or the code deletes the key. Delete or update the key whenever the row changes (write-through), keep a TTL as a safety net, and never use the cache for values that must be exact.

Many shops use both: replicas for general reads, the cache for the hottest pages, and the same ElastiCache cluster (or a separate one) for sessions.

#Stage 7: Multi-AZ for every data tier

The web tier already spans AZs. A data tier with one copy in one AZ would undo that: when its AZ fails, every server is healthy and the shop is still down.

• RDS Multi-AZ: a standby in another AZ receives every write synchronously. If the primary fails, RDS points the same DNS endpoint at the standby, usually within 60 to 120 seconds, with no committed data lost. The standby serves no reads; replicas do that (lesson 7.1).
• ElastiCache Multi-AZ (Valkey or Redis OSS): a replica in another AZ, with automatic failover. If the primary node fails, ElastiCache promotes the replica, usually within seconds, and the primary endpoint follows it. Replication is asynchronous, so the last instant of writes can be lost, which is acceptable for sessions. ElastiCache Serverless replicates across AZs for you.
• DynamoDB, if it holds the sessions, is already stored in three AZs: nothing to configure.

Note what Multi-AZ on each tier buys during an RDS failover: for a minute or two, checkouts wait and retry, but browsing and carts keep working, because the sessions are in a different store that did not fail.

#Stage 8: security groups chained tier to tier

The last requirement is that only the right tier can reach the data. Security groups can name another security group as the source of a rule (lesson 3.5), which chains the tiers together:

Why this is the strongest form:

• Only the ALB faces the internet. Nobody can bypass it to reach a web server, and nobody on the internet can reach the database port.
• Membership, not addresses. Any instance with app-sg can reach the data tier, so instances launched by Auto Scaling work at once, with no IP list to update. Any instance without it is refused, even inside the same VPC and subnet.
• Stateful. Replies flow back automatically; no outbound or return rules are needed for these connections.
• Layered. Put the web and data tiers in private subnets with no public IP, and keep RDS “not publicly accessible”. The security groups and the subnet design each stop an attacker on their own.

A rule that allows the database port from 0.0.0.0/0, or from the whole VPC CIDR, quietly breaks the chain: any instance in the VPC can then reach RDS.

#The three-tier picture

Put the stages together and you have the classic three-tier web application, the target of this whole lesson:

Lesson 9.3 adds the next piece: shared files (uploaded images) with EFS, and launching new instances fast.

#What goes wrong

#How this fits system design

• Push state to the edges of the stateless tier: in the client (small, signed, identity only), or in a shared store (everything else). The web tier itself holds nothing it cannot lose.
• Pick the store by the data’s lifetime: seconds to hours of session state in a cache or DynamoDB with TTL; years of orders in a relational database with backups.
• Scale reads before writes: replicas and caches are cheap compared with sharding the primary. When writes outgrow one instance, the next steps are Aurora (lesson 7.3) or a different data model, covered in a later chapter on choosing databases.
• Every tier needs its own failure plan: Multi-AZ for the data, several AZs for the web tier, health checks at the ALB.
• Security groups describe the architecture: reading the four rules above tells a reviewer exactly who talks to whom.

#Exam keywords → the next step

#Try it yourself

The commands use made-up names: target group tg-shop, security groups sg-0alb…, sg-0app…, sg-0db…, sg-0cache….

# Stage 2: duration-based stickiness on the target group (1 hour)
aws elbv2 modify-target-group-attributes --target-group-arn <tg-shop-arn> \
  --attributes Key=stickiness.enabled,Value=true Key=stickiness.type,Value=lb_cookie \
               Key=stickiness.lb_cookie.duration_seconds,Value=3600

# Stage 4a: a session store on ElastiCache (Valkey) with a replica in another AZ
aws elasticache create-replication-group --replication-group-id shop-sessions \
  --replication-group-description "My Cloth sessions" --engine valkey \
  --cache-node-type cache.t4g.small --num-cache-clusters 2 \
  --automatic-failover-enabled --multi-az-enabled \
  --cache-subnet-group-name private-data --security-group-ids sg-0cache1111

# Write and read a session with a 30-minute TTL (from a web server)
redis-cli -h <primary-endpoint> --tls SET sess:7f3a '{"cart":["shirt","trousers"]}' EX 1800
redis-cli -h <primary-endpoint> --tls TTL sess:7f3a

# Stage 4b: or a DynamoDB session table with TTL on the expires_at attribute
aws dynamodb create-table --table-name shop-sessions \
  --attribute-definitions AttributeName=session_id,AttributeType=S \
  --key-schema AttributeName=session_id,KeyType=HASH --billing-mode PAY_PER_REQUEST
aws dynamodb update-time-to-live --table-name shop-sessions \
  --time-to-live-specification Enabled=true,AttributeName=expires_at

# Stage 6: a read replica for product pages
aws rds create-db-instance-read-replica --db-instance-identifier shop-db-replica-1 \
  --source-db-instance-identifier shop-db

# Stage 7: make the primary Multi-AZ
aws rds modify-db-instance --db-instance-identifier shop-db --multi-az --apply-immediately

# Stage 8: chain the security groups
aws ec2 authorize-security-group-ingress --group-id sg-0app2222 \
  --protocol tcp --port 80 --source-group sg-0alb1111
aws ec2 authorize-security-group-ingress --group-id sg-0db3333 \
  --protocol tcp --port 3306 --source-group sg-0app2222
aws ec2 authorize-security-group-ingress --group-id sg-0cache1111 \
  --protocol tcp --port 6379 --source-group sg-0app2222

To see stickiness from the client side, request the site twice with curl -i https://shop.example.com/ -c jar.txt -b jar.txt and look for the AWSALB cookie in the response headers.

#Summary

Key takeaways

• A stateful app needs data from earlier requests. With state in server RAM, a load balancer that spreads requests loses the cart on every hop.
• Sticky sessions keep a shopper on one server with a cookie. No code change, but carts die with the instance (crash, scale-in, deployment) and load becomes uneven. Right answer only for a legacy app whose code cannot change. Session ID vs session data: lesson 6.4.
• Cart in a cookie: stateless servers, but about 4 KB per cookie, sent with every request, and the client can change it, so it must be signed and validated and never hold secrets.
• Server-side sessions: the cookie holds only a random session ID; the data lives in ElastiCache (microseconds, Valkey/Redis OSS with a replica) or DynamoDB (durable, serverless, TTL deletes within a few days). A TTL expires idle sessions. Needs a code change.
• User data (accounts, addresses, orders) goes in RDS; the web tier keeps nothing.
• Read scaling: read replicas (small code change, any query, replica lag) or ElastiCache lazy loading (code change, microseconds, stale until TTL or invalidation). “No code change” → replicas.
• Multi-AZ for RDS (synchronous standby, 60–120 s failover) and ElastiCache (replica, failover in seconds) lets the data tier survive an AZ failure; DynamoDB already spans three AZs.
• Security groups reference each other: internet → alb-sg (80/443), alb-sg → app-sg, app-sg → cache-sg (6379) and db-sg (3306). Never open the data ports to 0.0.0.0/0.
• Together: the classic three-tier web application.

Self-check: a stateful web application30 questions