Two ways to get an answer
For any derived value you can compute it fresh on each request or cache the result and serve the stored copy. Caching trades memory and freshness for speed.
When caching wins
- The computation is expensive and the inputs change rarely, so the same answer is reused many times.
- A high hit rate means most requests skip the work entirely.
When caching hurts
- If inputs change constantly, the cache is stale or churns, and you pay for both storage and recompute.
- Caching adds invalidation complexity, the hard problem of knowing when a stored value is no longer correct.
- A cache adds a new failure mode, like a stampede when many requests miss at once and all recompute together.
Picking a strategy
- Cache values with high reuse and slow change, like a rendered page or a config blob.
- Recompute values that are cheap or change on every request.
- Bound the cost of misses with techniques like request coalescing so a stampede recomputes once, not a thousand times.
Cache when reuse is high and change is slow, otherwise just compute.
Key idea
Caching trades memory and freshness for speed and only pays off when results are reused often and change slowly, while adding invalidation and stampede risks.