Design a Distributed Cache - Redis Architecture from Scratch

Caching is the single most effective technique for improving system performance. A well-designed distributed cache can reduce database load by 90%+ and cut response times from hundreds of milliseconds to sub-millisecond.

Step 1 - Requirements Gathering

Functional Requirements

• Store key-value pairs with optional TTL (time-to-live)
• Support common operations: GET, SET, DELETE, and bulk operations
• Distribute data across multiple nodes for horizontal scaling

Non-Functional Requirements

• Low latency - sub-millisecond reads for cache hits
• High throughput - 100K+ operations per second per node
• High availability - cache failures shouldn't bring down the system
• Scalability - add nodes without downtime

When to Cache

• Read-heavy workloads - when reads vastly outnumber writes
• Expensive computations - ML model inference results, complex aggregations
• Frequently accessed data - user sessions, product catalogues, configuration

Step 2 - Caching Strategies

Cache-Aside (Lazy Loading)

The most common pattern. Application checks the cache first; on a miss, it reads from the database, then populates the cache:

GET user:123 → Cache HIT → return cached data
GET user:456 → Cache MISS → query DB → SET cache → return data

Pros: only caches what's actually requested. Cons: cache misses incur latency; data can become stale.

Read-Through

The cache itself handles database reads on a miss. The application only talks to the cache layer, which abstracts the data source.

Write-Through

Every write goes to both the cache and the database synchronously. Ensures cache is always consistent but adds write latency.

Write-Behind (Write-Back)

Writes go to the cache immediately; the cache asynchronously flushes to the database in batches. Lower write latency but risks data loss if the cache node crashes before flushing.

| Strategy | Consistency | Read Latency | Write Latency | Use Case | |----------|-------------|-------------|---------------|----------| | Cache-Aside | Eventual | Miss penalty | N/A | General purpose | | Read-Through | Eventual | Miss penalty | N/A | Simplified reads | | Write-Through | Strong | Fast | Slower | Consistency-critical | | Write-Behind | Eventual | Fast | Fast | Write-heavy workloads |

Step 3 - Eviction Policies

When the cache is full, which entries do we remove?

• LRU (Least Recently Used) - evicts the entry that hasn't been accessed for the longest time. Best general-purpose policy
• LFU (Least Frequently Used) - evicts the entry accessed least often. Better for skewed access patterns
• TTL-based - entries expire after a set time regardless of access. Essential for data freshness
• Random - surprisingly effective and very simple to implement

Step 4 - Consistent Hashing for Distribution

The Problem with Simple Hashing

With hash(key) % N nodes, adding or removing a node remaps nearly all keys, causing a massive cache miss storm.

Consistent Hashing with Virtual Nodes

Consistent hashing maps both keys and nodes onto a ring. Each key is assigned to the next node clockwise on the ring:

Hash Ring (0 to 2^32):
     Node A (3 virtual nodes: A1, A2, A3)
     Node B (3 virtual nodes: B1, B2, B3)
     Node C (3 virtual nodes: C1, C2, C3)

Key "user:123" → hash → lands between B2 and A1 → assigned to Node A

Virtual nodes solve the uneven distribution problem. Each physical node gets multiple positions on the ring, ensuring balanced load even with few physical nodes.

When a node is added or removed, only keys adjacent to it on the ring are remapped - approximately K/N keys (where K is total keys and N is node count), not all of them.

Step 5 - Cache Stampede Prevention

A cache stampede occurs when a popular cached entry expires and hundreds of concurrent requests simultaneously miss the cache and hit the database.

Solutions

Mutex/Locking: When a cache miss occurs, acquire a distributed lock. Only one request fetches from the database; others wait for the cache to be repopulated.

Probabilistic Early Expiry: Each request has a small random chance of refreshing the cache before the TTL actually expires. This spreads refresh load over time rather than creating a thundering herd at expiry.

Background Refresh: A background process refreshes popular cache entries before they expire. The cache is never empty for hot keys.

Step 6 - Redis Cluster Architecture

Redis Cluster is the most widely deployed distributed cache:

• 16,384 hash slots distributed across master nodes
• Each key maps to a slot via CRC16(key) % 16384
• Replication: each master has one or more replicas for failover
• Automatic failover: if a master fails, its replica is promoted within seconds

Slots 0-5460:     Master A ←→ Replica A'
Slots 5461-10922: Master B ←→ Replica B'
Slots 10923-16383: Master C ←→ Replica C'

Monitoring Key Metrics

• Hit ratio - target 95%+; below 80% indicates a caching problem
• Memory usage - stay below 80% to avoid evictions
• Eviction rate - high evictions mean the cache is undersized
• Latency percentiles - p99 should be under 1ms for cache hits

Step 7 - Real-World Caching Patterns

| Pattern | Implementation | TTL | |---------|---------------|-----| | Session store | SET session:{id} {data} | 30 minutes | | API response cache | SET api:/users/123 {json} | 5 minutes | | Database query cache | SET query:{hash} {result} | 1 minute | | Rate limiter counter | INCR ratelimit:{user}:{window} | Window duration | | Feature flags | HSET features {flag} {value} | 10 seconds |

Cache Invalidation

The hardest problem in computer science (alongside naming things):

• TTL-based - simple, tolerates staleness for the TTL duration
• Event-driven - publish invalidation events when data changes (via Kafka or Redis Pub/Sub)
• Version-based - include a version in the cache key; increment version on writes to effectively invalidate old entries

📚 Further Reading

• Consistent Hashing - ByteByteGo - visual explanation of consistent hashing and virtual nodes
• Redis Cluster Specification - official Redis Cluster architecture documentation
• Caching Strategies - AWS - practical caching patterns for cloud applications