ElastiCache and Caching Patterns

By the end of this lesson you will be able to

• Explain what an in-memory cache is, why it makes reads fast and the database quiet, and what it costs you (code changes and stale data).
• Choose between the three ElastiCache engines, Valkey, Redis OSS and Memcached, from an exam scenario.
• Describe the topologies: cluster mode disabled and enabled, ElastiCache Serverless, and Global Datastore, with their limits.
• Secure a cache with security groups, AUTH tokens, RBAC users, IAM authentication, TLS and KMS, and SASL for Memcached.
• Apply lazy loading, write-through, TTLs and session stores, and handle invalidation, eviction and cache stampedes.
• Use sorted sets, pub/sub, counters and geospatial indexes for leaderboards, chat, rate limiting and “nearby” searches.
• Tell ElastiCache (a cache) from MemoryDB (a durable in-memory database), and recognise every exam keyword for both.

#Why put a cache in front of the database?

#Memory is a thousand times faster than a query

An RDS or Aurora database keeps its data on storage and does real work for every query: parse the SQL, plan it, read pages, maybe join and sort. A well-tuned query answers in a few milliseconds. When thousands of users ask for the same data again and again (the home page, a popular product, a user profile), the database does that same work again and again.

Amazon ElastiCache is a fully managed in-memory data store. It keeps data in RAM instead of on disk, as simple keys and values, and answers a read in microseconds (well under a millisecond). AWS runs the nodes for you: setup, patching, monitoring, failure recovery and backups. In the same way that RDS gives you a managed relational database, ElastiCache gives you managed Valkey, Redis OSS or Memcached.

It is used for two main jobs:

#The trade-off: the application must change

A cache is not transparent. The application has to be cache-aware: its code decides when to read the cache, what to do on a miss, when to write to the cache and when to delete a key. That is the key difference from an RDS read replica, which you use simply by pointing reads at another endpoint.

Two other costs come with any cache:

• Stale data. The cache holds a copy. When the database changes, the copy may be out of date until it is refreshed or expires. The application needs an invalidation strategy so it serves current data where that matters.
• Another component to size, secure and monitor. A cache that disappears suddenly sends every read to the database at once (more on that under “cache stampede”).

#The engines: Valkey, Redis OSS and Memcached

ElastiCache runs three engines. The notes and most exam questions speak of Redis and Memcached; since 2024 AWS also offers Valkey and renamed its Redis engine Redis OSS.

• Redis OSS (short for Remote Dictionary Server) is the open-source Redis, up to version 7.x on ElastiCache. It is more than a cache: an in-memory data store with rich data structures, replication and snapshots.
• Valkey is the open-source fork of Redis 7.2, created in 2024 after Redis changed its licence, and run by the Linux Foundation. It speaks the same protocol and commands, so Redis clients and code work unchanged. AWS now recommends Valkey as the default: it is about 20% cheaper on nodes and 33% cheaper on Serverless than the other engines, with a minimum Serverless storage charge of 100 MB instead of 1 GB.
• Memcached is a much simpler, older cache: plain keys and values, many threads, nothing else.

In this lesson “Redis” in an exam question means Valkey or Redis OSS: every Redis answer applies to Valkey too.

When to choose which:

• Valkey or Redis OSS when you need high availability (Multi-AZ, automatic failover), read replicas, persistence / backup and restore, complex data types (leaderboards, queues, geo), pub/sub, or a session store that must survive a node failure. That is almost every case, which is why the exam’s default “cache” answer is Redis.
• Memcached when you need the simplest possible cache of plain objects, want multi-threaded performance on large nodes, want to add and remove nodes freely, and can afford to lose cached data (it can always be rebuilt from the database).

#Topologies: how the nodes are arranged

#Cluster mode disabled: one shard

With cluster mode disabled, a Valkey or Redis OSS cluster (a replication group in the API) has exactly one shard: one primary node that takes reads and writes, and 0 to 5 read replicas that copy it asynchronously.

• The primary endpoint always points to the current primary; send writes there.
• The reader endpoint spreads connections across the replicas; send reads there.
• Put replicas in other AZs and turn on Multi-AZ with automatic failover: if the primary fails, ElastiCache promotes a replica, usually within seconds, and updates the primary endpoint’s DNS. The last few writes may be lost because replication is asynchronous.
• You scale reads by adding replicas and scale memory and writes only by choosing a bigger node type (vertical scaling).

#Cluster mode enabled: many shards

With cluster mode enabled, the key space is split into 16,384 hash slots spread over several shards. Each shard has its own primary and up to 5 replicas.

• Writes scale horizontally: each shard’s primary takes writes for its own slots.
• The default quota is 90 nodes per cluster, which you can raise to 500 nodes (Valkey or Redis OSS 5.0.6 and later). Examples: 500 shards with no replicas, or 83 shards of 1 primary + 5 replicas (83 × 6 = 498).
• You can add or remove shards online (online resharding).
• Automatic failover is required.
• Clients must be cluster-aware, and a multi-key command only works when all its keys live in the same slot.

Memcached has its own simple form of sharding: a cluster of up to 60 nodes, and the client hashes each key to one node. There is no replica of any node.

#ElastiCache Serverless

ElastiCache Serverless (since November 2023) removes node sizing entirely. You give the cache a name and get one endpoint; ElastiCache scales memory, compute and network up and out on its own.

• Works with Valkey, Redis OSS and Memcached; runs in cluster mode behind a proxy, so clients just need TLS.
• Data is replicated across multiple AZs automatically, with a 99.99% availability SLA.
• Always encrypted in transit and at rest.
• You pay for data stored (GB-hours) and ElastiCache Processing Units (ECPUs): a simple read or write of up to 1 KB costs 1 ECPU, a 3.2 KB write costs 3.2 ECPUs, and CPU-heavy commands cost more. In us-east-1 Valkey Serverless costs about $0.084 per GB-hour and $0.0023 per million ECPUs.

Choose Serverless for new, spiky or unpredictable workloads and when you do not want to plan capacity; choose node-based clusters when you want full control of node types, parameter groups and the lowest cost for steady, predictable load.

#Global Datastore

Global Datastore (Valkey and Redis OSS, node-based clusters) replicates a cluster to up to two other Regions. The primary cluster takes writes; the secondary clusters are read-only copies, updated asynchronously, usually within a second. Use it for low-latency local reads worldwide and cross-Region disaster recovery: if the primary Region fails, you promote a secondary cluster to primary.

#Security

Security for a cache works in layers, the same way as for RDS (lesson 7.4):

1. Network: VPC and security groups. Node-based caches run in your VPC, in the private subnets of a cache subnet group, and have no public IP. The cache’s security group allows the cache port (6379 for Valkey/Redis OSS, 11211 for Memcached) only from the application tier’s security group, exactly as you allow EC2 to reach RDS. This is the answer to “stop other EC2 instances from reaching the cache”.
2. IAM for the API. IAM policies decide who may call the ElastiCache API: create, modify, delete clusters, take snapshots. They do not, by themselves, control who can read or write keys.
3. Authentication for data access (Valkey / Redis OSS):
   • Redis AUTH token: a password set on the cluster; every client must send it (AUTH <token>) before running commands. It is an extra layer on top of security groups and requires in-transit encryption to be on.
   • RBAC users and user groups (Redis OSS 6.0+, Valkey): separate users with their own passwords and access strings, for example read-only on session:* keys. Replaces the single shared token.
   • IAM authentication (Valkey 7.2+, Redis OSS 7.0+): the application signs a short-lived IAM token (valid 15 minutes) with its IAM role and sends it instead of a password; the IAM policy needs elasticache:Connect. Requires TLS; IAM connections are dropped after 12 hours unless re-authenticated. No password to store or rotate.
4. Encryption. In transit with TLS between clients and nodes and between nodes; at rest with KMS for disks, snapshots and swap, using an AWS managed key or your own customer managed key.
5. Memcached supports SASL authentication (version 1.5.16+): a username and password per client. It has no AUTH token, RBAC or IAM authentication on node-based clusters.

#Caching patterns

The pattern decides when data enters the cache, when it leaves, and how stale it may get.

#Lazy loading (cache-aside)

1. Read the cache first

   The application asks the cache for the key, for example user:7.

2. Hit: answer at once

   If the key is there (a cache hit), the application returns it. The database does nothing.

3. Miss: read the database, then fill the cache

   If the key is not there (a cache miss), the application queries the database, returns the result to the user and writes it into the cache so the next read hits.

• Pros: only data that someone actually requests is cached, so memory is not wasted. A failed or empty cache node is not fatal: reads just miss and go to the database (slower, but correct).
• Cons:
  • Miss penalty: a miss costs three round trips (cache get → database query → cache set), so the first read is slower than reading the database directly.
  • Stale data: when the database is updated, the cache is not; readers see the old value until the key expires or is deleted.
• Fix: give every key a TTL (time to live, for example 1 hour). Stale data then lives at most one TTL. Many applications also delete the key whenever they update the row, so the next read reloads it.

How much does the hit ratio matter? With a hit ratio h, the database receives (1 − h) of all reads, and the average read latency is h × (cache trip) + (1 − h) × (cache trip + database query + cache trip). At 10,000 reads per second and a 90% hit ratio, only 1,000 reads per second reach the database. With a 0.5 ms cache and a 10 ms query, the average read takes 0.9 × 0.5 + 0.1 × 11 = 1.55 ms instead of 10 ms. With a cold cache (0% hits) every read takes 11 ms: slower than no cache.

#Write-through

Whenever the application writes to the database, it also writes the new value to the cache, in the same code path.

• Pros: the cache is never stale for data written this way, and reads of that data always hit. Users always see the latest value.
• Cons:
  • Write penalty: every update is two writes, so writes are slower. Users usually accept this because writes are rarer than reads and expected to take a moment.
  • Cache churn: every written item lands in the cache, including data nobody will ever read, which wastes memory and can push out useful keys.
  • Missing data: a new or replaced node, or data written before the cache existed, is not in the cache until it is written again.

#Combine them, and add a TTL

The two patterns fix each other’s weaknesses, so production systems usually use both:

• Lazy loading fills the cache on misses (new nodes, old data).
• Write-through keeps data fresh when it changes.
• A TTL on every key removes data nobody reads (churn) and limits how long any missed update can stay stale.

Two smaller patterns are worth knowing by name: write-behind (write to the cache now, flush to the database later, fast but risky) and refresh-ahead (reload hot keys shortly before they expire).

#Cache invalidation is the hard part

As programmer Phil Karlton put it: “There are only two hard things in Computer Science: cache invalidation and naming things.”

The difficulty is knowing when a cached copy has stopped being true. Practical rules:

• Prefer deleting a key on update over overwriting it; the next read reloads the truth.
• Always set a TTL, even on write-through keys, as a safety net.
• Pick the TTL from how stale the business can tolerate: seconds for stock levels and prices, minutes for profiles, hours for a product catalogue.
• Never cache data that must be exact at every moment (an account balance at payment time); read it from the database.

#Eviction: when memory is full

A cache node has a fixed amount of RAM. When it is full and a new key arrives, the maxmemory policy decides what to remove:

Memcached always evicts by LRU. Watch the CloudWatch metrics Evictions and DatabaseMemoryUsagePercentage: steady evictions mean the cache is too small (scale up or out) or keys have no TTL. With volatile-lru, keys without a TTL are never evicted, so a cache full of them ends in out-of-memory errors.

#Cache stampede (thundering herd)

When a hot key expires, or a cache node is replaced and comes back empty, hundreds of requests can miss at the same instant and all run the same expensive query. The database, which was sized for the 10% of reads that normally miss, suddenly gets all of them. Defences:

• One rebuilder: the first request takes a short lock (SET lock:home 1 NX EX 5) and reloads the key; the others wait briefly or serve the old value.
• Jitter: add a random few seconds to each TTL so keys do not all expire together.
• Refresh ahead of expiry for the hottest keys, and warm a new cache before sending it traffic.
• Keep a replica so a node failure is a failover, not an empty cache.

#Session store

Sessions are the classic second job for ElastiCache. The application writes the session under a random ID (sess:a1f) with a TTL equal to the session timeout; the browser holds only the ID in a cookie; every web server reads the session from the cache. Each request resets the TTL, so active users stay logged in and idle sessions expire on their own.

This is how lesson 6.4’s problem (sticky sessions lose the cart when a server dies) is solved properly. For sessions, use Valkey or Redis OSS with a replica and Multi-AZ (or Serverless), because losing the session store logs out every user; use DynamoDB when sessions must be fully durable.

#What else Valkey and Redis OSS are good at

Because Valkey and Redis OSS have real data structures and run each command atomically, they solve problems that a database or Memcached handles badly.

#Gaming leaderboards with sorted sets

Ranking millions of players by score is expensive in a relational database: an ORDER BY score DESC on every page view. A sorted set keeps members unique and always ordered by score. Each new score is ranked in real time, in O(log N) time.

• ZADD game:lb 4200 rafi adds or updates a player.
• ZINCRBY game:lb 50 rafi adds points.
• ZREVRANGE game:lb 0 2 WITHSCORES returns the top 3.
• ZREVRANK game:lb rafi returns one player’s rank.

#Pub/sub for chat and notifications

With pub/sub, publishers send messages to a channel (PUBLISH room:42 "hi") and every connected subscriber (SUBSCRIBE room:42) receives them at once. It fits real-time chat, live notifications and broadcasting changes to many servers. Messages are fire-and-forget: a subscriber that is offline misses them. When messages must be kept and replayed, use streams (or SQS, Kinesis).

#Rate limiting with counters

INCR is atomic, so many servers can count against one key safely. A fixed-window limiter of 100 requests per user per minute:

1. INCR rate:user42:202610251430 (the key includes the current minute).
2. On the first increment, EXPIRE the key after 60 seconds.
3. If the counter is above 100, reject the request with HTTP 429.

#Geospatial: “show nearby drivers”

Apps like Uber or Foodpanda need “drivers or restaurants within 2 km of me”. GEOADD drivers 90.4125 23.8103 d17 stores a driver’s position; GEOSEARCH drivers FROMLONLAT 90.41 23.81 BYRADIUS 2 km ASC returns the nearest ones in microseconds.

#Distributed locks (brief)

SET lock:order:991 <owner> NX PX 10000 succeeds for only one client, so only one worker processes order 991; the lock expires on its own if the worker crashes. Good for avoiding duplicate work; for correctness-critical locking, a database or DynamoDB conditional write is safer.

#MemoryDB: when the in-memory store is the database

Amazon MemoryDB is a durable, Valkey- and Redis OSS-compatible primary database. It looks like Redis to your code, but every write is committed to a Multi-AZ transactional log before it is acknowledged, so no acknowledged write is lost even if nodes fail.

Exam keywords: “durable in-memory database”, “Redis-compatible primary database”, “microsecond reads and no data loss” → MemoryDB. “Cache”, “reduce database load”, “session state” → ElastiCache.

#Caching in system design

• Cache the hot, repeated, slightly-stale-is-fine reads: product pages, profiles, configuration, results of expensive queries. Read from the database directly when data must be exact.
• Default pattern: lazy loading + delete on write + a TTL on every key; add write-through for data that is read right after it is written.
• Valkey with a replica in another AZ (or Serverless) for anything you cannot afford to lose suddenly: sessions, leaderboards, rate-limit counters. Memcached only for disposable object caches.
• Plan for the empty cache: size the database so it survives a cache failover, add jitter, warm caches, and protect hot keys from stampedes.
• Layer caches: CloudFront at the edge, ElastiCache in front of the database, RDS read replicas or Aurora replicas for reads that need SQL. DynamoDB has its own built-in cache, which you will meet in the DynamoDB lesson.
• Watch CacheHitRate, Evictions, CurrConnections, EngineCPUUtilization and ReplicationLag in CloudWatch.

For the general idea of sessions and state, see the System Design lesson State, cookies and sessions.

#Try it yourself

# 1. The quickest start: a Serverless Valkey cache (encrypted, Multi-AZ, no sizing)
aws elasticache create-serverless-cache --serverless-cache-name app-cache \
  --engine valkey --security-group-ids sg-0cache0123456789 \
  --subnet-ids subnet-0aaa1111 subnet-0bbb2222
aws elasticache describe-serverless-caches --serverless-cache-name app-cache \
  --query "ServerlessCaches[0].[Status,Endpoint.Address]"

# 2. A node-based Valkey cluster: 1 primary + 1 replica in another AZ, failover, TLS, KMS, AUTH token
aws elasticache create-replication-group --replication-group-id sessions \
  --replication-group-description "session store" \
  --engine valkey --cache-node-type cache.r7g.large \
  --num-cache-clusters 2 --multi-az-enabled --automatic-failover-enabled \
  --cache-subnet-group-name private-cache --security-group-ids sg-0cache0123456789 \
  --transit-encryption-enabled --at-rest-encryption-enabled \
  --auth-token 'use-a-long-random-secret-here' \
  --snapshot-retention-limit 7

# 3. Test a failover (promotes the replica)
aws elasticache test-failover --replication-group-id sessions --node-group-id 0001

# 4. Security group: only the app tier may reach port 6379
aws ec2 authorize-security-group-ingress --group-id sg-0cache0123456789 \
  --protocol tcp --port 6379 --source-group sg-0app0123456789

From an EC2 instance in the app tier (the endpoint name is an example):

valkey-cli -h master.sessions.abc123.use1.cache.amazonaws.com -p 6379 --tls -a "$AUTH_TOKEN"

SET user:7 '{"name":"Rina","email":"rina@example.com"}' EX 300   # cache for 5 minutes
GET user:7                                                       # hit
TTL user:7                                                       # seconds left
DEL user:7                                                       # invalidate on update

ZADD game:lb 4200 rafi 3900 mitu 3650 arif                       # leaderboard
ZINCRBY game:lb 500 arif
ZREVRANGE game:lb 0 2 WITHSCORES                                 # top 3

INCR rate:user42:202610251430                                    # rate limiting
EXPIRE rate:user42:202610251430 60

CONFIG GET maxmemory-policy        # on ElastiCache, change it in the parameter group instead
INFO stats                         # keyspace_hits, keyspace_misses, evicted_keys

Lazy loading with a TTL in Python:

import json, valkey

cache = valkey.Valkey(host=ENDPOINT, port=6379, ssl=True, password=AUTH_TOKEN)

def get_user(user_id):
    key = f"user:{user_id}"
    hit = cache.get(key)                      # trip 1
    if hit:
        return json.loads(hit)
    row = db.query("SELECT * FROM users WHERE id = %s", user_id)   # trip 2
    cache.set(key, json.dumps(row), ex=300)   # trip 3, TTL 5 minutes
    return row

def update_email(user_id, email):
    db.execute("UPDATE users SET email = %s WHERE id = %s", email, user_id)
    cache.delete(f"user:{user_id}")           # invalidate: next read reloads

#Summary

Key takeaways

• ElastiCache is a managed in-memory store: microsecond reads, used as a database cache (offload read-heavy workloads) and a session store (stateless web tier). It needs application code changes, unlike read replicas, and can serve stale data.
• Engines: Valkey (open-source Redis fork, AWS’s recommended default, about 20% cheaper on nodes and 33% on Serverless), Redis OSS, and Memcached. Valkey/Redis OSS: replication (up to 5 replicas per shard), Multi-AZ automatic failover, snapshots (no AOF on ElastiCache), rich data types, pub/sub. Memcached: multi-threaded, sharding only, no replication, no persistence.
• Cluster mode disabled: 1 shard, 1 primary + up to 5 replicas, scale up for writes. Cluster mode enabled: many shards, 90 nodes by default and up to 500. Serverless: pay for GB-hours and ECPUs, Multi-AZ, always encrypted. Global Datastore: up to 2 more Regions, read-only secondaries, DR.
• Security: VPC + security groups for the network; IAM for the API; Redis AUTH token (needs TLS), RBAC users, or IAM authentication (Valkey 7.2+ / Redis OSS 7.0+) for data access; TLS and KMS; SASL for Memcached.
• Lazy loading: cache only what is read, 3-trip miss penalty, stale data. Write-through: never stale, write penalty, churn, missing data on new nodes. Combine both and put a TTL on everything.
• Invalidation is the hard part: delete on update, always set a TTL. Eviction follows the maxmemory policy (volatile-lru by default; LRU, LFU, TTL, random). Protect hot keys from stampedes with a lock, jitter and refresh-ahead.
• Valkey/Redis use cases: leaderboards (sorted sets), pub/sub chat, rate limiting (INCR + EXPIRE), geospatial “nearby”, simple distributed locks.
• MemoryDB = durable Redis-compatible primary database with a Multi-AZ transaction log; ElastiCache = cache.

Self-check: ElastiCache and caching patterns30 questions