A feed is built in stages
Serving a modern feed is a pipeline of distinct stages, each narrowing and improving a list of candidate posts. Splitting it this way lets each stage scale and evolve on its own.
The stages
- Retrieval gathers candidates. It reads the timeline cache and pulls fresh posts from followed accounts, producing a large rough set.
- Filtering drops what should not appear, already seen posts, blocked authors, and policy violations.
- Ranking scores the surviving candidates with a model and orders them by predicted engagement.
- Re ranking applies business rules on top, diversity so one author does not dominate, and injected items like ads.
- Serving cuts the ranked list to one page and returns it with a cursor.
Why a pipeline
- Each stage has a clear contract, a list in and a smaller better list out.
- Stages can use different infrastructure, retrieval is data heavy, ranking is compute heavy.
- Teams can improve ranking without touching retrieval.
The pipeline turns a vague request for a good feed into a sequence of well defined, testable steps.
Key idea
A feed is generated by a pipeline of retrieval, filtering, ranking, re ranking, and serving, each stage narrowing a candidate list so the parts scale and evolve independently.