⚡ ~/naveed Interview Prep
⚡ Portfolio Home 👤 About Naveed Bio & Credentials 📫 Contact Naveed Get In Touch ✍️ Engineering Blog Deep Dives 🎯 Interview Hub 1,000+ Scenarios ☸️ Kubernetes Mastery Hub 24 Modules 🎮 DevOps Arcade & Quizzes Subnet Blitz ⚡ 🗺️ DevOps Roadmaps PDFs & Guides 🤖 Morpheus Analysis AI Quant ↗ 🛠️ Developer Tools Utilities 🧪 Labs & Experiments 📄 Interactive CV & Certs 🔗 All Links & Socials ⚡ Join The Dispatch (Weekly SRE Newsletter) →
← Back to All Databases & Storage Interview Questions Scenario 41 of 54 in Databases & Storage
Staff Distributed Systems Architect Database Distributed Caching & High-Performance SRE System Design
🎯 Target Role / Context: Principal Distributed Systems Architect / Staff Performance SRE System Design Interview

Q: Your high-throughput read API serves 300,000 requests per second. Querying central Redis for every request saturates Redis network bandwidth (10 Gbps limit). Storing a local L1 cache inside each Go/Java application pod delivers 0.01ms latency, but pods serve stale data when entities are updated in the database. How do you design a multi-tier cache hierarchy with sub-millisecond distributed cache invalidation across 200 application pods?

Engineering a high-performance two-tier cache hierarchy (L1 in-memory application cache + L2 shared Redis cluster) handling 300,000 RPS with sub-millisecond invalidation broadcasts via Redis Pub/Sub.

#System Design #Caching #Cache Invalidation #Redis #Pub/Sub #Cache-Aside #Two-Tier Cache
🎙️ Candidate Opening & Architectural Context
"Centralized caches hit network bandwidth scaling limits under massive read loads. We architected a two-tier caching platform combining local L1 in-memory pod caching (Go Ristretto / Java Caffeine), shared L2 Redis Cluster, and distributed invalidation broadcasting over Redis Pub/Sub."
Advertisement
⚡ Recommended Practice Lab

Want to master this scenario in a live sandbox? KodeKloud's PostgreSQL Database Administration & High Availability Course covers this exact problem with hands-on terminal drills.

🛠️ Production Runbook & Step-by-Step Resolution

Direct Answer: Centralized caches hit network bandwidth scaling limits under massive read loads. We architected a two-tier caching platform combining local L1 in-memory pod caching (Go Ristretto / Java Caffeine), shared L2 Redis Cluster, and distributed invalidation broadcasting over Redis Pub/Sub.

1️⃣

Architect Two-Tier Cache Topology: L1 In-Memory + L2 Redis Cluster

Achieve nanosecond read latency while offloading central Redis network traffic:

  • Tier 1 (L1 In-Memory): Each application pod allocates 500 MB RAM to a local in-memory cache (Caffeine for JVM / Ristretto for Go) using TinyLFU eviction policies.
  • Tier 2 (L2 Shared Cache): Shared multi-AZ Redis Cluster caching complete serialized entities with a 24-hour TTL.
  • Read Flow: Application checks L1 (0.005ms) -> if miss, checks L2 Redis (1.2ms) -> if miss, queries PostgreSQL database (15ms), populating both L2 and L1.
Pro Tip: L1 cache absorbs 88% of all read requests, slashing Redis network bandwidth consumption from 12 Gbps down to 1.4 Gbps.
2️⃣

Broadcast Sub-Millisecond Invalidation via Redis Pub/Sub

Evict stale local pod memory the instant an update occurs:

  • Write Flow: When an update modifies product:1234, the writing pod updates PostgreSQL, deletes the key from L2 Redis, and publishes an invalidation event: PUBLISH cache:invalidate 'product:1234'.
  • Pod Subscription: All 200 application pods maintain an active subscription to cache:invalidate.
  • Local Eviction: Upon receiving the broadcast message, every pod immediately purges product:1234 from its local L1 memory in < 1 millisecond.
Pro Tip: Broadcasting invalidations via Pub/Sub ensures that local L1 pod caches never serve stale data longer than a few milliseconds.
Advertisement
3️⃣

Mitigate Pub/Sub Packet Drops via Monotonic Versioning & Short TTL

Prevent silent data desynchronization if a pod temporarily loses network connection:

  • Short L1 TTL: All L1 cache entries enforce a maximum background TTL of 60 seconds (even if never invalidated), bounding maximum staleness to 60s under catastrophic network splits.
  • Monotonic Entity Versioning: Objects embed an incremental version counter: { 'id': 1234, 'version': 48 }. If an L1 cache entry has a version lower than an incoming write request header, the local entry is evicted immediately.
Pro Tip: Redis Pub/Sub does not guarantee message delivery to disconnected subscribers; bounding L1 TTLs to 60 seconds guarantees eventual convergence.
4️⃣

Prevent Cache Stampedes via SingleFlight Mutex Suppression

Shield the underlying database during sudden cache eviction spikes:

  • SingleFlight Pattern: When an L1/L2 cache miss occurs for a popular key, the pod uses Go singleflight.Group (or Java synchronized loader) to ensure that only ONE worker thread queries the database.
  • Shared Result: All 500 concurrent in-flight requests wait for the single database query to complete and share the result in memory.
  • Database Impact: Database CPU utilization during cache invalidations dropped from 95% spikes down to a steady 12%.
Pro Tip: SingleFlight mutex suppression prevents the classic 'thundering herd' cache stampede from crashing the database when a popular cache key is invalidated.
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"A two-tier cache hierarchy combines local pod L1 in-memory caches to absorb 88% of reads, L2 Redis Cluster for shared persistence, Redis Pub/Sub for sub-millisecond invalidation broadcasts, and SingleFlight to prevent database stampedes."
⚡ 60-Second Elevator Pitch Talking Points
  • Deploy L1 in-memory pod caching (TinyLFU) backed by an L2 Redis Cluster to slash network bandwidth.
  • Broadcast real-time cache invalidation messages across all pods in < 1ms via Redis Pub/Sub.
  • Enforce short 60s L1 safety TTLs and monotonic entity versioning to handle network blips.
  • Use SingleFlight request coalescing to eliminate thundering herd database stampedes.
⛔ Critical Candidate Mistakes to Avoid in This Scenario

Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:

  1. Applying exclusive-lock DDL table migrations on live production databases during peak traffic windows when configuring System Design Multi DDL migrations that acquire an exclusive table lock block all concurrent queries, queuing connections until the database connection pool exhausts.
  2. Allowing high-frequency queries to trigger full table scans without appropriate composite indexing when configuring System Design Multi Sequential table scans saturate database CPU and disk I/O, evicting frequently queried data from the buffer cache and causing cascading timeouts.
  3. Disabling referential integrity constraints during bulk ETL imports without post-import consistency validation when configuring System Design Multi Skipping relational consistency checks risks introducing orphaned records and referential integrity violations into production tables.
  4. Relying on database snapshot backups that have never been verified through automated restoration drills when configuring System Design Multi Backups that have never been tested with automated restore verification frequently fail during real disaster recovery scenarios.
Advertisement
Want more Databases & Storage scenarios?
Explore our complete collection of scenario-based Databases & Storage interview runbooks.
Browse All Databases & Storage Questions →