Caching Layers: Where to Put Them and What They Cost

Caching is the most reliable performance tool available and the one most likely to produce a subtle bug six months later. Both facts come from the same source: a cache is a second copy of the truth, and second copies drift.

This is a tour of the layers where you can put one, what each buys, and the bill each sends.

Before you cache anything

Caching is frequently reached for before the actual problem has been identified, which means the cache becomes a permanent workaround for something that could have been fixed properly in an afternoon.

The order that saves you the most grief:

1. Measure. Find where the time actually goes. It is regularly not where the team assumed.
2. Fix the query. A missing index or an N+1 is a two-hour fix that removes the problem permanently. Caching it instead preserves it forever behind a layer of complexity.
3. Reduce the work. Can you return fewer rows, serialise fewer fields, avoid the call entirely?
4. Then cache. Once the operation is as cheap as it can reasonably be and it is still too slow or too frequent.

The layers, outermost first

Order matters. A request answered further out is answered more cheaply, because every layer it skips is work nobody has to do.

1. Browser cache

The cheapest possible cache hit, because the request never leaves the device. Controlled entirely by response headers.

The pattern that works: fingerprint static assets with a content hash in the filename and serve them with Cache-Control: public, max-age=31536000, immutable. Because the filename changes whenever the content does, you never need to invalidate anything. For HTML, use a short max-age or no-cache with an ETag, so the browser revalidates cheaply and gets a 304 when nothing changed.

2. CDN

A shared cache close to the user, geographically. Obvious for static assets, and underused for HTML.

Two techniques are worth knowing. Stale-while-revalidate serves the cached copy immediately while refreshing in the background, so users never wait for a revalidation. Surrogate keys let you tag responses and purge everything carrying a tag in one call — which is what makes CDN-caching dynamic pages practical rather than terrifying.

The trap is caching a personalised page for everyone. Vary the cache key on whatever makes the response different, or make sure logged-in traffic bypasses the CDN entirely. Getting this wrong serves one user’s account page to another, and it is a genuinely common incident.

3. Reverse proxy

Nginx, Varnish or your load balancer, caching in front of the application. Conceptually the same as a CDN but under your control and inside your network, which makes it useful for internal services and for responses too dynamic to push to the edge but too expensive to regenerate constantly.

4. Application-level fragment cache

Caching rendered pieces of a page. Rails popularised the pattern properly with Russian doll caching: nest fragments, key each by the record’s identifier and its updated_at, and let outer fragments include the keys of inner ones.

The elegance is that nothing is ever invalidated. Touching a record changes its timestamp, which changes its cache key, which changes the key of every fragment containing it. Old entries are never purged — they simply become unreachable and get evicted eventually. Key-based expiry like this is the most maintainable caching strategy available, and it is worth structuring your models to permit it.

5. Object cache

Redis or Memcached, holding computed values: a serialised object, an expensive aggregate, a rendered template, a rate-limit counter.

This is the most flexible layer and the one where most caching bugs live, because the key namespace is yours to design and therefore yours to get wrong. Some discipline that pays off:

• Namespace keys consistently and predictably, so you can reason about and purge groups of them.
• Include a version component in the key prefix, so a deploy that changes the serialisation format invalidates everything by moving to a new namespace.
• Always set a TTL, even on entries you intend to invalidate explicitly. It bounds the blast radius of a bug in your invalidation logic.
• Never let the cache be the only copy. If Redis restarts empty, the system should be slow, not broken.

6. Query cache

Caching the result set of a specific query. Useful for expensive aggregates that are read constantly and change rarely — report totals, dashboard counts, leaderboards.

Note that databases also cache internally: buffer pools, plan caches, materialised views. Before adding a query cache in your application, check whether a materialised view refreshed on a schedule solves the same problem inside the database, where consistency is somebody else’s job.

7. Database and disk cache

Mostly automatic, but worth understanding because it explains a common surprise: a query that is slow in isolation and fast in a loop is being served from the buffer pool. Benchmark accordingly, and be suspicious of any measurement taken on a warm cache that will run cold in production.

Invalidation strategies

Phil Karlton’s line about cache invalidation being one of the two hard problems in computer science endures because it is true. There are three broad approaches and their maintenance costs differ enormously.

Prefer key-based where the data model allows it, TTL where it does not, and explicit purging only where staleness is genuinely unacceptable. Explicit purging degrades over time in a predictable way: the invalidation call is written correctly once, and then a new code path updates the record without it.

The failure modes worth knowing

• Thundering herd. A popular key expires and a thousand concurrent requests all miss and all recompute simultaneously. Fix with a lock so one request regenerates while others serve stale, or by adding jitter to TTLs so keys do not expire in lockstep.
• Cache stampede on cold start. An empty cache after a Redis restart or a deploy means every request is a miss at once. Warm critical keys deliberately, and make sure the system degrades to slow rather than to down.
• Unbounded growth. No TTL and no eviction policy means memory climbs until something is killed. Configure a maxmemory policy explicitly.
• Caching personalised data under a shared key. The most damaging failure on this list, because it is a data leak rather than a performance problem.
• Caching errors. An upstream returns a 500, you cache the response, and now every user gets a cached error for the next hour. Only cache successful responses.

Boolean Solutions experience with caching

We introduce caching into applications and, at least as often, remove it from applications where it was accumulated rather than designed. The single most common finding is a cache layer added to work around a query that could have been fixed with an index — and which is now load-bearing, so removing it requires fixing the query anyway plus untangling everything built on top.

If your application is slow and you want to know whether caching is the answer before you commit to it, get in touch.

Further reading

• Prevent unnecessary network requests with the HTTP Cache — the clearest explanation of cache headers
• Caching with Rails — the best available write-up of key-based expiry, useful even outside Rails
• Redis eviction policies — what happens when memory fills up, which you should decide rather than discover
• Modern Web Architectures: Part 1 — the architectural context these layers sit inside