Caching Patterns for Scalable Systems: Edge → Reverse Proxy → Redis (with Stampede Protection)

Caching is not a performance trick. It's an architecture decision about where you want load to terminate. Most scalable systems are fast because the primary datastore is protected—not because application code is magically efficient. This guide explains a layered caching model (Edge → Reverse Proxy → Redis) and how to keep it safe with stampede protection.

Caching is also one of the easiest ways to create subtle correctness bugs and incident loops. The failure modes are predictable: cache stampedes, stale reads, incorrect cache keys, and invisible hotspots. If you design for those failure modes up front, caching becomes a reliability feature—not an outage trigger.

The core idea: terminate load as early as possible

Every request that reaches your database is a request you chose not to absorb earlier. Layered caching reduces tail latency variance, reduces pressure on shared resources, and creates headroom for spikes. But the system only stays healthy if caching is designed as a controlled subsystem with explicit keys, TTLs, and failure behavior.

Layer 1: Edge caching (CDN)

Edge caching delivers the cheapest wins because it scales with the internet, not with your cluster. Use it to absorb volume before it ever touches origin.

Use edge caching when

• Responses are identical across many users (marketing, docs, static assets, public catalog pages).
• You can define safe cache keys (path/query/headers) and tolerate bounded staleness or revalidation.
• You want global latency improvements without increasing origin complexity.

Common edge failures

• Personalization leaks: user-specific responses cached with incorrect keys (wrong Vary behavior).
• Cache poisoning: unsafe caching of dynamic content through relaxed headers.
• Incorrect assumptions: "it's public" until one header makes it not public.

Rule: if a response is user-specific, assume it's unsafe to edge-cache unless you can prove the cache key is correct.

Layer 2: Reverse proxy caching

Reverse proxy caching (Nginx, Envoy, API gateway) protects origin services when edge caching is not possible, or when you want an additional buffer before app and database tiers.

Use reverse proxy caching when

• You need a cache close to services (auth'd traffic, internal APIs, limited edge control).
• You want centralized policy: consistent TTLs, cache keys, and request collapsing.
• You want "serve stale on origin trouble" behavior (when configured explicitly).

Common reverse proxy failures

• Fragmentation: high-cardinality keys (user IDs, random query params) destroy hit rate.
• Inconsistent behavior: local caches per proxy lead to uneven hit rates and confusing tails.
• Invisible caching: no hit/miss instrumentation means you can't reason about behavior under load.

Layer 3: Redis caching

Redis is where caching becomes a real system: shared state, high QPS, and real operational failure modes. It's powerful because it allows many application instances to share cached data reliably.

Use Redis caching when

• Multiple app instances need the same cached values with low latency.
• You need explicit cache-aside control in application logic.
• You want targeted invalidation and TTL-backed staleness control.

Common Redis failures

• Hot keys: one key dominates, saturates Redis, and spreads latency across the fleet.
• Stampedes: many workers rebuild the same value at once and hammer the origin.
• Memory pressure: evictions create "random" latency spikes and cache effectiveness collapse.

Rule: Redis is not free. If you don't design keys, TTLs, and stampede controls, Redis becomes your next bottleneck.

Cache-aside vs write-through

Most production systems should default to cache-aside. Read path: check cache → on miss, read source → populate cache → return. Write path: write source of truth → invalidate or update cache.

Write-through can work, but it increases coupling and makes failure handling harder. If cache writes fail, do you fail the request or accept inconsistency? Cache-aside keeps the source of truth authoritative and makes failure behavior easier to reason about.

Stampede protection

Cache stampede is one of the most common ways caching causes incidents. It happens when a popular key expires and many workers miss simultaneously, creating a rebuild spike that you created yourself. Stampede protection ensures a miss does not create unbounded fan-in.

Techniques that hold up in production

• Request coalescing (single-flight): only one worker rebuilds; others wait briefly or serve stale.
• Stale-while-revalidate: serve stale values while one worker refreshes in the background.
• Jittered TTLs: spread expirations across time to avoid synchronized misses.
• Negative caching: cache "not found" for short TTLs to prevent repeated misses.
• Pre-warm for known spikes: for launches/flash sales, warm safely rather than relying on on-demand rebuilds.

Key design and hot keys

Key design determines whether caching scales. If one key dominates, you've created a shared resource under extreme contention—Redis becomes a latency amplifier.

Practical key design rules

• Keep keys low-cardinality when possible; avoid per-user keys for shared data.
• Version keys for safe invalidation (e.g., product:v3:1234).
• Define TTL policy explicitly; avoid unbounded growth.
• Control payload size; don't store huge objects by default.

Common hot key sources

• Global "top items" lists loaded by everyone.
• Popular tenant or a single shared dashboard cache key.
• Feed endpoints where everyone hits the same shard/key at the same moment.

Invalidation strategy

Most caching failures are correctness failures. You need an explicit stance on staleness:

• Strict correctness: invalidate on write + short TTL + consistency-sensitive reads avoid caches.
• Bounded staleness: TTL-based reads where short staleness is acceptable.
• Event-driven invalidation: publish invalidation events when state changes across services.

Practical advice: if reliable invalidation is hard, prefer bounded staleness and make product behavior tolerate it.

Observability

Caching without visibility is a trap. At minimum, you need signals that explain whether caching is effective and whether it is becoming a bottleneck.

• Hit rate by endpoint and key-group (not a single global number)
• p95/p99 latency for cache operations
• Redis CPU/memory, evictions, network throughput
• Miss spikes and rebuild time
• Single-flight contention / lock wait time (if used)
• Origin load when cache is "hot" (to detect fragmentation or stampede leakage)

How to validate caching actually helps

Don't ship caching as a belief. Validate it as an intervention. Success means improved tails and reduced saturation, not just higher hit rate.

Before/after checks

• p95/p99 latency for the target endpoints
• DB QPS and DB saturation during peak
• Error rate under spikes
• Behavior at TTL boundaries (do tails jump?)
• Cache latency distribution (does cache become the bottleneck?)

Final takeaway

Layered caching—Edge → Reverse Proxy → Redis—is a scalable pattern because it terminates load early and protects shared resources. But it only stays safe when you design for its failure modes: stampedes, hot keys, correctness drift, and invisibility.

If you want help turning caching into a measurable scalability win, start by baselining tails and saturation, then validate before/after. Get an AI system audit.