← Lessons

quiz vs the machine

Gold1470

System Design

Circuit Breaking in the Mesh

Stopping calls to an unhealthy service so failures do not cascade across the fleet.

5 min read · core · beat Gold to climb

Containing Failure

A circuit breaker stops a service from hammering a downstream dependency that is already failing. In a mesh, the proxy enforces this so the protection is consistent everywhere.

How It Trips

  • The proxy tracks errors and slow responses to a destination.
  • When failures cross a threshold, the circuit opens and new calls fail fast.
  • After a cooldown, a few probe requests test if the destination recovered, the half open state.

Failing fast is the point. A request that would have timed out after thirty seconds is rejected instantly, freeing threads and connections.

Connection Limits

The mesh also caps concurrent connections and pending requests to a destination. Outlier detection ejects an individual unhealthy endpoint from the load balancing pool while keeping healthy ones in service.

Why It Saves the System

Without breaking, a slow dependency soaks up callers threads until they too stall, and the failure spreads. The breaker turns a slow cascade into a fast, contained, recoverable error.

Key idea

A mesh circuit breaker fails fast when a downstream is unhealthy and ejects bad endpoints, turning a creeping cascade into a contained, recoverable error.

Check yourself

Answer to earn rating on the learn ladder.

1. What is the benefit of failing fast when a circuit is open?

2. What does outlier detection do?