Sizing the hot set
Cache sizing estimates how much memory holds the data that is accessed often enough to be worth caching, the hot set.
The Pareto starting point
Many workloads follow a rough 80 20 pattern: a small fraction of items serves most requests. So you often only need to cache the top 20 percent of items by access frequency, not everything.
- Estimate total items and bytes per item.
- Take the fraction that is hot, often around twenty percent.
- Multiply to get the working set size.
If 100 gigabytes of data is hot at twenty percent, you need roughly 20 gigabytes of cache.
Account for overhead
- Cache structures add per key overhead beyond the raw value.
- Leave headroom so eviction does not thrash near full capacity.
Under sizing the cache collapses the hit rate, sending traffic to the slow backing store and erasing the benefit.
Key idea
Cache memory targets the hot set, often the top twenty percent of items, plus overhead and headroom to keep the hit rate high.