Caching at Every Layer

By the end of this lesson you will be able to

• Draw the full caching path of a typical AWS application (client → CloudFront → API Gateway → Lambda or EC2 → ElastiCache or DAX → database, with S3 behind CloudFront) and say what each layer caches.
• Explain the one trade-off behind every layer: the closer to the user you cache, the faster and cheaper each request, but the harder the copy is to invalidate and the staler it can get.
• Configure and reason about each layer’s TTL and invalidation: CloudFront TTLs and invalidations, the API Gateway stage cache (REST only, TTL, cache keys, client bypass with Cache-Control: max-age=0), ElastiCache TTLs and delete-on-write, the DAX item TTL.
• Decide when a cache saves money and when it only adds a bill.
• Choose lazy loading, write-through or both on this path, and name the data that must never be cached.
• Pick DAX or ElastiCache, and keep session state out of the web tier.
• Read an exam scenario’s keywords and name the caching layer it wants.

#Why cache at every layer

Every earlier chapter added one cache: CloudFront in lesson 11.1, ElastiCache in lesson 7.5, DAX in lesson 15.5, the API Gateway stage cache in lesson 15.6. This lesson does not teach them again. It puts them on one request path and asks the architect’s questions: which layer should answer which request, how long may each copy live, what happens to a price change on its way through four caches, and which data must never be cached at all.

Caching is used at many layers for three reasons, and the exam uses all three words: to make the application faster (lower latency), to reduce load on the database and the compute behind it, and to save cost.

#The request path of a flash sale

Think of a Daraz-style shop on the night of an 11.11 flash sale. Shoppers in Dhaka, Chattogram and Sylhet open the app; the shop runs in the Singapore Region.

#The one trade-off: close is fast, far is fresh

Six words sum up the whole picture: caching, TTL, network, computation, cost, latency. Every layer trades them against each other in the same direction:

• Closer to the user (the edge): the request saves the network trip to the Region, all the computation behind it and the database query. Lowest latency, lowest cost per request. But the copy sits in hundreds of places, so it is the hardest to invalidate and can be the stalest.
• Closer to the database (ElastiCache, DAX): each request still pays for the network and the compute in front, but the cache is in one place your code controls, so invalidating a key is one command and the data can be kept almost fresh.
• Staleness adds up. If the API cache keeps a response for 60 s and the Valkey key it was built from had a 60 s TTL, a shopper can see an old price for almost two minutes. The worst case is the sum of the TTLs of every layer that holds a copy.

So the design question for each kind of data is: how far up the path can it go before its staleness hurts? A product photo can go all the way to the edge; a product list can sit at the API for a minute; a price can sit in Valkey if it is deleted on change; the stock count of the last 40 phones cannot be cached anywhere.

#Layer 1: the edge (CloudFront)

Recap (lessons 11.1 and 11.2): CloudFront is AWS’s global CDN. It keeps copies at more than 750 edge locations (PoPs) close to users, with larger Regional Edge Caches behind them. A request is answered by the nearest PoP; only a miss goes back to the origin (S3, an ALB, EC2, API Gateway or any HTTP server).

What matters on the full path:

• TTL. How long a copy counts as fresh before CloudFront checks the origin again. The origin’s Cache-Control: max-age decides it within the minimum and maximum TTL of the cache policy; with no header the default TTL is 24 hours (86,400 s). When the TTL ends, CloudFront revalidates or fetches the new version.
• Cache key. By default the domain and URL path. Add to it (through the cache policy) exactly the query strings, headers and cookies that change the response, and nothing else: a missing value serves the wrong copy, an extra one ruins the hit ratio.
• Behaviours per path. Static files under /img/* use a long-TTL policy such as CachingOptimized; dynamic /api/* usually uses CachingDisabled, so the edge passes it on. Even then the edge helps: TLS ends near the user and the request travels on the AWS network.
• Invalidation. create-invalidation removes paths from every edge before the TTL ends. The first 1,000 paths a month are free, then 0.005 USD per path (a wildcard path counts as one). The better habit for files that change is a versioned file name (banner-v2.jpg, app.3f9a1c.js): a new name is a new cache key, nothing to invalidate, and it even beats browser caches that an invalidation cannot reach.
• S3 behind CloudFront. Keep the bucket private and let only the distribution read it with Origin Access Control (OAC). Data from AWS origins to CloudFront is free, so serving S3 through CloudFront is usually cheaper as well as faster.

#Layer 2: the API (API Gateway stage cache)

Recap (lesson 15.6): a REST API stage can have a cache. When a request’s key is in it, API Gateway answers without calling the backend. On the full path this layer sits behind the edge and in front of all your compute: a hit here saves the Lambda invocation (or the EC2 work) and everything behind it.

The facts the exam and production both care about:

• REST APIs only. HTTP APIs and WebSocket APIs have no built-in cache; put CloudFront with a caching policy in front if you need one.
• Per stage, with a size from 0.5 GB to 237 GB. Creating or resizing it takes a few minutes, and resizing empties it.
• TTL: default 300 s, from 0 (off) up to 3,600 s. Per method you can override the TTL or turn caching off.
• GET only by default. When you enable the stage cache, only GET methods are cached; other methods need an explicit override (and rarely should be cached).
• Cache key: the method and path, plus the cache key parameters you choose (query strings, headers, path parameters). Forget a parameter that changes the response and one shopper gets another shopper’s answer.
• A response larger than 1 MB is not cached. Cached data can be encrypted. Watch the CloudWatch metrics CacheHitCount and CacheMissCount.
• Flush the whole stage cache from the console or with flush-stage-cache; the backend then takes the full load until the cache refills.

#Client bypass: Cache-Control: max-age=0

A client that needs a fresh answer can send Cache-Control: max-age=0. API Gateway then calls the backend and replaces the cached entry with the new response. Who may do this is controlled by two settings on the stage or method:

1. Require authorization (requireAuthorizationForCacheControl). When it is on, the caller needs an IAM permission for the action execute-api:InvalidateCache on that method’s resource ARN. When it is off, any client can bypass and refresh the cache, and AWS warns that this can raise latency for everyone.
2. Unauthorized request handling (unauthorizedCacheControlHeaderStrategy) for callers without the permission:
   • FAIL_WITH_403: the request is refused with 403.
   • SUCCEED_WITH_RESPONSE_HEADER: the header is ignored, the cached answer is served, and a warning header is added.
   • SUCCEED_WITHOUT_RESPONSE_HEADER: the header is ignored silently.

#Does the API cache save money?

The notes’ exam trick says API caching can “reduce API Gateway cost by limiting backend calls”. Read it carefully:

• The cache is billed per hour by size, whether it is used or not, and is not in the free tier: in us-east-1 about 0.02 USD an hour for 0.5 GB (about 14.60 USD a month) up to 3.80 USD an hour for 237 GB (about 2,770 USD a month).
• Every request is still an API Gateway request. A cache hit is billed like a miss (3.50 USD per million REST requests for the first tier). The API Gateway line of the bill never goes down.
• What goes down is the backend: Lambda invocations and duration, EC2 capacity, database load.

A worked example. A catalogue API gets 200,000 GET requests a month, served by a 128 MB Lambda function that runs 100 ms. Lambda costs 0.20 USD per million requests plus 0.0000166667 USD per GB-second: 0.04 USD + 200,000 × 0.1 × 0.125 × 0.0000166667 ≈ 0.04 + 0.04 = 0.08 USD a month. A 0.5 GB cache costs about 14.60 USD. Caching here makes the API faster but the bill 182 times bigger.

Now the flash sale: 50 million requests a month with a 90% hit ratio. The cache saves 45 million invocations: 9 USD of requests + 45,000,000 × 0.1 × 0.125 × 0.0000166667 ≈ 9.38 USD of duration, about 18 USD, plus millions of database queries that would otherwise need a bigger database. Here the 14.60 USD cache pays for itself. Caching reduces cost only when backend savings beat the cache fee.

#Layer 3: the application cache (ElastiCache)

Recap (lesson 7.5): ElastiCache is an in-memory layer your code puts in front of the database (RDS, Aurora, DynamoDB, anything), so the database does not run the same query again and again. Unlike the layers above, it caches whatever your code decides: a query result, a computed product page, a session, a leaderboard.

#Which engine

The notes describe two versions, Redis and Memcached. Since 2024 ElastiCache runs three engines:

• Valkey: the open-source fork of Redis, run by the Linux Foundation, same commands and clients. AWS’s recommended default, and cheaper than the others.
• Redis OSS: the open-source Redis engine (what the exam still calls “Redis”).
• Memcached: the simple, older cache.

Plus ElastiCache Serverless (for all three engines): no nodes to size, data replicated across AZs, billed by data stored and processing units. Everything the exam says about “Redis” applies to Valkey.

#Lazy loading and write-through on this path

Lesson 7.5 teaches both strategies step by step. Here is what they mean when the cache sits in the middle of a four-layer path.

• Lazy loading (cache-aside). The app reads the cache first; on a hit it answers, on a miss it reads the database, answers, and writes the result into the cache. Only data someone asks for is cached, so memory is saved, and a failed or empty node is not fatal (reads just go to the database). The costs: a miss penalty (three trips: cache get, database query, cache set, so the first read is slower than no cache at all), and stale data, because a database update does not touch the cache.
• Write-through. Every write to the database is followed by a write to the cache. Reads of written data always hit and are never stale. The costs: a write penalty (two writes per update), cache churn (data nobody will read fills memory), and missing data on a new node, which stays empty for a key until that key is written again.

The practical answer is to combine them: lazy loading so any miss refills the cache (new nodes, old data), write-through or delete-on-write so changed data is fresh, and a TTL on every key so data nobody reads leaves and any missed update is bounded. When memory is full, the eviction policy removes keys (the default volatile-lru evicts the least recently used key among keys with a TTL), which is one more reason every key should have a TTL.

#What never goes in a cache

Some data must be exact at the moment it is used. Caching it anywhere, at any TTL, turns a speed problem into a correctness problem:

• Stock counts and seats during a sale: read them from the database, and make the purchase a conditional write (decrement only if stock > 0).
• Account balances at the moment of a payment, as in a bKash-style wallet: a balance can be cached for the home screen, never for the debit decision.
• Per-user data on a shared key: a cart or profile cached at the API layer under a key without the user is shown to the next user.
• One-time tokens, OTPs and anything security-sensitive.

#Layer 3 for DynamoDB: DAX

Recap (lesson 15.5): DynamoDB answers in single-digit milliseconds. DynamoDB Accelerator (DAX) is an in-memory cache cluster built only for DynamoDB that answers cached reads in microseconds and takes read load off the table.

• API-compatible. DAX speaks the DynamoDB API. You swap the DynamoDB client for the DAX client and point it at the cluster endpoint; GetItem, Query and PutItem stay the same. The notes’ “no code rewrite” is right, with the precision that the client and endpoint change (a small change, not zero).
• Read-through and write-through: misses are read from the table and cached; writes through DAX go to the table and then update the item cache.
• Eventually consistent reads only. A strongly consistent read (ConsistentRead=true) or a transactional read passes straight through to the table and is not cached.
• Default TTL 5 minutes for both the item cache and the query cache. The query cache (results of Query and Scan) is not updated by writes.
• Writes that bypass DAX (another application, an import job, a global-table replica) leave the cache stale until the TTL ends.
• Runs as a cluster in your VPC (one primary and up to 10 read replicas; three nodes in three AZs for production), billed per node-hour. It helps reads, not writes.

#Session state: keep the web tier stateless

Recap (lessons 9.1, 9.2 and 6.4): session data, such as who is logged in and what is in the cart, must not live only on one EC2 instance. Save it in an external store and give the browser only a session ID. The notes call this session caching.

• ElastiCache for Valkey or Redis OSS: sub-millisecond reads, a TTL per session equal to the session timeout; use a replica with Multi-AZ, because losing the store logs everyone out.
• DynamoDB: durable and serverless, with a TTL attribute so expired sessions delete themselves. Good when sessions must survive anything.
• Memcached also works if losing sessions on a node failure is acceptable.

Sticky sessions (ALB stickiness cookies) are the shortcut, not the fix. They keep a user on one instance, so a cart in memory survives only while that instance lives: a scale-in, a failed health check or a deployment loses it. They also spread load unevenly: a new instance from a scale-out receives only new users, while the busy instance keeps its stuck users. Sessions in a shared store remove both problems; the web tier becomes stateless, and any instance can serve any request.

#Two cache-like alternatives

• RDS read replicas (lesson 7.1) take read queries off the primary without any cache code: send reports and read-only pages to the replica endpoint. They are copies of the whole database, updated asynchronously (seconds of lag), and they still run every query; a cache returns a stored answer without running anything. Choose replicas when reads are varied (reporting, search pages) and a little lag is fine; choose ElastiCache when the same few results are read again and again.
• S3 behind CloudFront with OAC: for static files the “cache layer” is simply the edge, with S3 as the private origin. No ElastiCache is needed for images, scripts or downloads.

#What to cache where

#End-to-end example: an 11.11 flash sale

The Daraz-style shop’s design for 11.11, layer by layer:

1. Edge: product images and scripts

   All images and scripts live in a private S3 bucket behind CloudFront with OAC, under versioned names with a one-year TTL. They never reach the Region again after the first request per edge.

2. API: the product list for 60 seconds

   GET /api/products?cat=… is cached at the REST API stage for 60 s with cat and page as cache key parameters. During the sale this one setting turns thousands of identical requests a minute into one Lambda run per category per minute. Require authorization for cache control, so only the admin app can force a refresh.

3. Application: prices in Valkey

   The Lambda functions read prices from ElastiCache for Valkey with lazy loading and a short TTL, and the seller’s price update deletes the key. The Valkey cluster has replicas in another AZ, so a node failure is a failover, not an empty cache and a database stampede.

4. Sessions: carts in DynamoDB

   Carts and logins live in a DynamoDB table keyed by session ID with a TTL attribute. Any function or instance can serve any shopper.

5. Never cached: the stock count

   The stock of the 40 sale phones is read from the database and each purchase is a conditional write. A few milliseconds slower, and never oversold.

Try the design in the tracer below: switch each layer on and off, change the TTL and the time since the seller’s last change, and watch where each request is answered, how long it takes and whether the answer can be wrong.

#What goes wrong

• Stacked TTLs. Each layer’s TTL looked harmless alone; together a price change takes minutes to reach shoppers. Add up the TTLs of every layer that holds the same data, and invalidate from the bottom up (database, Valkey key, API stage, edge).
• A cache key without the user. A per-user response cached under a shared key shows one user’s cart or profile to another. Turn caching off for per-user methods or put the user in the key.
• A cache key without a parameter that changes the response: the laptops page shows phones.
• Open cache bypass. “Require authorization” off on an API stage: any client can send max-age=0 on every request and the backend gets the full load.
• Default TTL surprises. CloudFront keeps objects without Cache-Control for 24 hours; a price list changed at noon is old until tomorrow.
• Cold cache after a node failure. A lazy-loaded cache with no replica comes back empty, every read misses, and the database is hit by a stampede. Use replicas and Multi-AZ, jitter on TTLs.
• Paying for an idle cache. An API stage cache for an API with few repeated requests costs more than it saves.
• DAX for the wrong reads. Strongly consistent reads, write-heavy tables and writes that bypass the cluster gain nothing from DAX.
• Sticky sessions as the session store. The cart disappears with the instance.

#Caching in system design

• Start from the data, not the service: for each kind of data, decide how stale it may be, and cache it as far up the path as that staleness allows.
• Static at the edge, shared at the API, computed in the app cache, exact in the database.
• Make every cache disposable: the system must stay correct (only slower) if any cache is empty. That rules out keeping the only copy of anything in a cache, and it is why the database stays the source of truth.
• Watch the hit ratio of each layer (CacheHitRate in CloudFront, CacheHitCount / CacheMissCount in API Gateway, CacheHits / CacheMisses and Evictions in ElastiCache). A cache with a low hit ratio costs money and adds a hop.
• The same path appears in lesson 24.1 for events and lesson 24.3 for blocking bad clients: the edge is also the cheapest place to stop traffic you do not want.

#Try it yourself

# 1. Edge: see whether CloudFront answered from cache, then invalidate one path
curl -sI https://d111111abcdef8.cloudfront.net/img/p881-v3.webp | grep -iE 'x-cache|age'
# x-cache: Hit from cloudfront
# age: 5312
aws cloudfront create-invalidation --distribution-id E1ABCDEXAMPLE \
  --paths "/api-docs/index.html"
# { "Invalidation": { "Id": "I2J0I21PCUYOIK", "Status": "InProgress", ... } }

# 2. API: provision a 0.5 GB stage cache and cache every GET for 60 s
aws apigateway update-stage --rest-api-id a1b2c3d4e5 --stage-name prod \
  --patch-operations op=replace,path=/cacheClusterEnabled,value=true \
                     op=replace,path=/cacheClusterSize,value=0.5 \
                     op=replace,path=/*/*/caching/enabled,value=true \
                     op=replace,path=/*/*/caching/ttlInSeconds,value=60
# { "stageName": "prod", "cacheClusterEnabled": true, "cacheClusterSize": "0.5",
#   "cacheClusterStatus": "CREATE_IN_PROGRESS", ... }

# 3. API: only callers with execute-api:InvalidateCache may bypass; others get 403
aws apigateway update-stage --rest-api-id a1b2c3d4e5 --stage-name prod \
  --patch-operations op=replace,path=/*/*/caching/requireAuthorizationForCacheControl,value=true \
                     'op=replace,path=/*/*/caching/unauthorizedCacheControlHeaderStrategy,value=FAIL_WITH_403'

# 4. API: turn caching off for the per-user cart method (path /cart is written ~1cart)
aws apigateway update-stage --rest-api-id a1b2c3d4e5 --stage-name prod \
  --patch-operations op=replace,path=/~1cart/GET/caching/enabled,value=false

# 5. API: empty the whole stage cache after an urgent price correction
aws apigateway flush-stage-cache --rest-api-id a1b2c3d4e5 --stage-name prod

# 6. Application: a serverless Valkey cache for prices
aws elasticache create-serverless-cache --serverless-cache-name prices \
  --engine valkey --subnet-ids subnet-0a1b2c3d subnet-0e5f6a7b \
  --security-group-ids sg-0123456789abcdef0
# { "ServerlessCache": { "ServerlessCacheName": "prices", "Status": "creating", "Engine": "valkey", ... } }

# 7. Lazy loading by hand with valkey-cli: miss, set with a TTL, hit, delete on write
valkey-cli -h prices-abc123.serverless.apse1.cache.amazonaws.com --tls GET price:881
# (nil)
valkey-cli -h prices-abc123.serverless.apse1.cache.amazonaws.com --tls SET price:881 1499 EX 60
# OK
valkey-cli -h prices-abc123.serverless.apse1.cache.amazonaws.com --tls TTL price:881
# (integer) 57
valkey-cli -h prices-abc123.serverless.apse1.cache.amazonaws.com --tls DEL price:881
# (integer) 1

# 8. DynamoDB: a three-node DAX cluster in front of the catalogue table
aws dax create-cluster --cluster-name catalogue-dax --node-type dax.r5.large \
  --replication-factor 3 --iam-role-arn arn:aws:iam::111122223333:role/DAXServiceRole \
  --subnet-group-name dax-private --security-group-ids sg-0123456789abcdef0
# { "Cluster": { "ClusterName": "catalogue-dax", "TotalNodes": 3, "Status": "creating", ... } }

# 9. Sessions: let DynamoDB delete expired sessions on its own
aws dynamodb update-time-to-live --table-name sessions \
  --time-to-live-specification Enabled=true,AttributeName=expiresAt
# { "TimeToLiveSpecification": { "Enabled": true, "AttributeName": "expiresAt" } }

# 10. Clean up the paid caches
aws apigateway update-stage --rest-api-id a1b2c3d4e5 --stage-name prod \
  --patch-operations op=replace,path=/cacheClusterEnabled,value=false
aws dax delete-cluster --cluster-name catalogue-dax
aws elasticache delete-serverless-cache --serverless-cache-name prices

Then watch the hit counts: in CloudWatch, CacheHitCount and CacheMissCount for the API stage, and CacheHits, CacheMisses and Evictions for the Valkey cache, while you repeat the same request and a different one.

#Summary

Key takeaways

• One path, four caches: CloudFront (edge) → API Gateway stage cache → ElastiCache or DAX (application/database) → database, with S3 behind CloudFront.
• The trade-off: the closer to the user, the lower the latency and cost per request, but the harder to invalidate and the staler. Staleness adds up across layers: worst case is the sum of the TTLs.
• CloudFront: default TTL 24 h, cache key and cache policy per behaviour, invalidation (1,000 paths free a month, then 0.005 USD per path) or better versioned file names; S3 origin private with OAC.
• API Gateway cache: REST APIs only, per stage, 0.5–237 GB, TTL default 300 s, max 3,600 s, GET only by default, cache key parameters, flush. Bypass with Cache-Control: max-age=0 needs execute-api:InvalidateCache only when require authorization is on. Billed per hour, and requests are still billed: it saves money only if backend savings beat the fee.
• ElastiCache: Valkey (recommended), Redis OSS, Memcached, plus Serverless. Redis for HA, Multi-AZ, backup, pub/sub, leaderboards; Memcached for simple, multi-threaded, purely horizontal key-value.
• Lazy loading: only requested data, but miss penalty and stale reads. Write-through: never stale, but write penalty, churn and empty new nodes. Combine both, TTL strongly recommended (not mandatory), delete on write.
• DAX: DynamoDB only, microseconds, API-compatible (swap client), eventually consistent reads only, default TTL 5 min, in your VPC.
• Sessions in ElastiCache or DynamoDB make the web tier stateless; sticky sessions lose state and spread load unevenly.
• Never cache stock counts, balances at payment time, OTPs, or per-user data on a shared key.

Self-check: caching at every layer31 questions