Requirements
- Post a short message and have it appear on followers home timelines quickly.
- Read a personalized home timeline ordered by recency.
- The follow graph is highly skewed, with a few accounts followed by millions.
High level design
The core decision is fanout on write versus fanout on read.
- Fanout on write: when a user posts, push the post id into a precomputed timeline cache for each follower. Reads are then a cheap cache lookup.
- Fanout on read: store posts per author and merge them at read time by pulling from each followed author.
- Hybrid: fan out writes for normal users, but for celebrity accounts pull their posts at read time and merge them in.
Bottlenecks
- Celebrity fanout: writing to millions of timelines per post is wasteful, so the hybrid model avoids it.
- Timeline storage: cache only the most recent few hundred ids per user and page back to the post store for older history.
- Ordering: merge by timestamp or a snowflake id so the merged feed stays roughly chronological.
Tradeoffs
- Fanout on write makes reads fast but writes expensive.
- Fanout on read keeps writes cheap but makes reads slow for users who follow many accounts.
Key idea
A timeline is a fanout problem where the skewed follow graph forces a hybrid of precomputed writes for normal users and read time merges for celebrities.