What shadow mode is
In shadow mode a candidate model receives a copy of live requests and produces predictions, but its output is logged not served. Users always see the production model. This tests the candidate on real traffic with zero user risk.
What it validates
- Operational health such as latency, errors, and resource use under real load.
- Prediction sanity by comparing candidate outputs to production outputs.
- Input handling to confirm the candidate processes real, messy traffic.
Shadow versus AB testing
Shadow mode answers can it run safely on production traffic. It does not measure business impact because no user ever sees its output. You typically run shadow first to derisk, then an AB test to measure value.
Why it is valuable
A candidate can pass offline tests yet fail on real inputs through timeouts, unseen categories, or memory spikes. Shadow mode surfaces these before any user is exposed.
Key idea
Shadow mode sends live traffic to a candidate whose predictions are logged not served, proving it runs safely on real inputs before users are exposed.