Requirements
- Turn features on or off without redeploying.
- Roll out to a percentage of users or specific segments.
- Evaluate flags with near zero latency in the application.
High level design
Flag definitions are managed centrally and evaluated locally in each service.
- Control plane: a dashboard and API define flags, targeting rules, and rollout percentages stored in a config database.
- Distribution: an SDK in each app pulls or streams the ruleset and caches it in memory.
- Evaluation: the SDK evaluates rules locally using a stable hash of the user id for consistent percentage rollouts.
Bottlenecks
- Evaluation latency: rules are evaluated locally so no network call is on the request path.
- Propagation delay: streaming or short polling pushes rule changes to SDKs within seconds.
- Consistent rollout: hashing the user id deterministically decides which bucket a user falls in.
Tradeoffs
- Local evaluation is fast but rule changes propagate with a small delay.
- Streaming cuts delay but costs more connections than polling.
Key idea
A feature flag service centralizes rule definitions but evaluates them locally in an SDK using a stable user hash, keeping evaluation off the network path.