Caching

"How would you make this read 100x faster?" In almost every design interview the winning first move is caching. It is the highest-leverage lever you have — and interviewers want to see that you also understand its costs.

The core idea

A cache is a small, fast store that keeps copies of frequently accessed data close to where it is needed, so you avoid a slow, expensive operation (a database query, an API call, a computation). Caches trade memory and freshness for speed. A request that finds its data in the cache is a cache hit; one that doesn't is a cache miss and must fall back to the source.

Beginner — where caches live

• Client/browser cache — closest to the user (HTTP cache headers).
• CDN — cached static content at edge locations near users.
• Application/in-memory cache — a shared store like Redis or Memcached holding hot query results and sessions.
• Database cache — the DB's own buffer pool.

The closer the cache is to the user, the bigger the latency win.

Intermediate — write strategies

How the cache stays in sync with the database defines the pattern:

• Cache-aside (lazy loading) — app checks cache; on a miss it reads the DB and populates the cache. Most common. Only requested data is cached.
• Write-through — writes go to cache and DB together; cache is always fresh but writes are slower.
• Write-back (write-behind) — writes go to cache first and flush to DB later; very fast writes but risk of data loss on crash.

Advanced — eviction and invalidation

A cache has limited size, so it needs an eviction policy: LRU (evict least-recently-used) is the default; others are LFU (least frequently used) and FIFO. Entries also carry a TTL (time to live) after which they expire. The hardest problem is invalidation — keeping cached data from going stale. Watch for a cache stampede (many simultaneous misses hammering the DB when a hot key expires); mitigate with staggered TTLs, locking, or refresh-ahead.

Worked example

A product page runs a heavy join taking 200 ms and is read 50,000 times/min. With cache-aside in Redis and a 60-second TTL, the first request is a miss (200 ms) and the rest are hits (~1 ms), cutting database load by well over 99% and dropping typical latency from 200 ms to a couple of milliseconds.

When to use

Cache read-heavy data that is expensive to produce and tolerant of slight staleness — product catalogs, user profiles, computed feeds. Avoid caching data that must always be perfectly fresh (e.g., a bank balance at the moment of a transaction) unless you invalidate carefully.

Tricks and mnemonic

Remember the trade with "caches make reads fast, but freshness and memory pay the bill." For the write patterns: aside = lazy, through = safe, back = fast-but-risky.

Fast revision of caching — the highest-leverage performance lever.

• A cache stores copies of hot data in fast storage to avoid slow operations.
• Hit = found in cache; miss = fall back to the source (DB/API).
• Layers (near→far): client/browser, CDN, app cache (Redis/Memcached), DB buffer.
• Write strategies: cache-aside (lazy, common), write-through (always fresh), write-back (fast, risk of loss).
• Eviction: LRU (default), LFU, FIFO; entries expire via TTL.
• Hardest problem is invalidation; beware cache stampede on hot-key expiry.
• Cache read-heavy, staleness-tolerant data; avoid for must-be-exact values.

Cover the answer, recall, then check. 8 cards.

Q1. What is a cache hit vs a cache miss?
A1. Hit = data found in the cache; miss = not found, so you fall back to the slower source.

Q2. What does caching trade away for speed?
A2. Memory (extra storage) and freshness (risk of stale data).

Q3. Describe the cache-aside pattern.
A3. App checks cache first; on a miss it reads the DB and populates the cache. Lazy loading.

Q4. Write-through vs write-back?
A4. Write-through writes to cache and DB together (fresh, slower); write-back writes to cache then flushes to DB later (fast, risk of loss).

Q5. What is the default eviction policy and what does it do?
A5. LRU — evicts the least-recently-used entry when the cache is full.

Q6. What is a TTL?
A6. Time to live — how long a cache entry stays valid before it expires.

Q7. What is a cache stampede?
A7. Many simultaneous misses hitting the DB when a hot key expires at once.

Q8. Name two common in-memory cache stores.
A8. Redis and Memcached.

Worked Example

Problem: A product page issues a database query that takes ~200 ms. It is called 10,000 times/minute, and 90% of calls hit the same 100 popular products. Design a cache and estimate the database-load reduction.

Solution: Add an in-memory cache (e.g. Redis) in front of the database using the cache-aside pattern: on a request, check the cache; on a miss, read the DB and populate the cache; set a TTL for freshness.

Because 90% of traffic targets a small hot set of 100 products, the cache hit ratio ≈ 90%:

• DB queries drop from 10,000/min to the ~10% misses ≈ 1,000/min — roughly a 90% reduction.
• Cached responses return in ~1-2 ms instead of 200 ms.

Handle staleness by invalidating or updating the cache entry when a product changes, and size the cache to hold the hot set comfortably.

Answer: ~90% fewer DB queries (10,000 → ~1,000/min) with near-instant hot reads.