Beyond local disk
A scraping server keeps only recent data on local disk. Long term storage moves older, immutable blocks to cheap object storage so years of history stay queryable without huge local disks.
How it is structured
- Recent data lives in time chunked blocks with an index of series to chunks.
- Sealed blocks are uploaded to object storage such as a bucket.
- A compactor merges small blocks into larger ones and applies downsampling tiers.
- A store gateway lets queries read blocks directly from object storage.
Querying across tiers
A query fans out to both recent local data and historical object storage blocks, then merges results. Block indexes and time based pruning keep scans bounded so old data does not slow every query.
Why this scales
Object storage is cheap, durable, and elastic. Separating compute from storage lets you scale queriers independently and run multiple tenants over one shared bucket of blocks.
Key idea
Long term storage seals metric blocks to cheap object storage, with a compactor downsampling and a store gateway serving them, separating compute from storage for durable scalable history.