The celebrity problem
Pure fan out on write breaks for celebrities. When an account with fifty million followers posts, the system must do fifty million feed writes at once, a spike that overwhelms the write path and delays the post.
Pure fan out on read breaks for normal users, since merging many followed accounts on every open is slow.
The hybrid answer
Most large systems use a hybrid. They split accounts by follower count:
- Normal authors use push. Their posts are fanned out to follower feeds at write time.
- Celebrity authors use pull. Their posts are not fanned out. Instead they are fetched and merged in at read time.
When a user opens their feed, the system reads their precomputed feed for normal authors and separately pulls recent celebrity posts the user follows, then merges the two streams.
Why it works
- It avoids the write storm from huge accounts.
- It keeps reads fast for the common case where most follows are normal.
- The merge cost is bounded by how many celebrities a user follows, which is small.
Key idea
A hybrid fan out pushes normal authors into feeds but pulls celebrity posts at read time, avoiding the write storm while keeping common reads fast.