The Scatter Gather Pattern
Scatter gather splits one request into many sub requests, sends them out in parallel, and then collects and combines the responses. It is the backbone of search engines, sharded databases, and any system that queries many sources at once.
The scatter step fans the work out. A search query might be sent to fifty index shards at the same time. The gather step waits for the responses and merges them into a single result, perhaps by sorting and taking the top matches.
- Scatter Broadcast the sub requests concurrently to all workers.
- Gather Collect responses and combine them into one answer.
- Aggregation Merging often involves sorting, summing, or deduplicating partial results.
The hard part is the gather. Total latency is governed by the slowest responder, not the average, because you cannot finish until enough answers arrive. This is the tail latency problem, and one slow shard can drag down the whole request.
Systems handle this with timeouts and partial results. If a shard does not answer in time, the gather proceeds with what it has and marks the result as incomplete rather than waiting forever. Some systems send the same sub request to two replicas and use whichever returns first.
Key idea
Scatter gather fans one request to many workers in parallel and merges their replies, with latency set by the slowest responder unless timeouts cut the tail.