On this page

Circuit Breaker Pattern

In electrical systems, a circuit breaker prevents a short circuit from burning down your house. In distributed systems, it prevents a failing dependency from taking down your entire service. The circuit breaker pattern wraps calls to external services and monitors failures. When failures exceed a threshold, the circuit "opens" and subsequent calls fail immediately without even attempting the call — giving the failing service time to recover instead of being hammered with requests it cannot handle.

Without circuit breakers, a single slow or failing dependency cascades through your entire system. Service A calls Service B, which is timing out. Service A's threads pile up waiting. Service A starts timing out. Service C, which calls Service A, starts timing out. Within seconds, your entire microservice mesh is a pile of threads waiting on connections that will never complete.

Why Cascades Happen

To see why the pattern is needed, follow the resources. A synchronous call holds something on the caller's side for as long as it is in flight: a thread, a goroutine and its stack, a connection from a pool, a slot in an in-memory request queue, or a file descriptor. When a dependency answers in 20 ms, each request holds those resources briefly. When the same dependency starts taking 20 seconds, which is usually the client timeout and not a real answer, each request holds them a thousand times longer.

Little's Law makes the arithmetic plain. The number of requests in flight equals arrival rate times time in system, . At 500 requests per second and 20 ms latency, a service has about 10 requests in flight to that dependency. At 20 seconds of latency it would need 10,000. No thread pool is sized for that, so the pool runs out, requests that have nothing to do with the slow dependency queue behind it, and the caller becomes slow for its own callers. The failure spreads up the call graph one hop at a time.

sequenceDiagram
    participant U as "Users"
    participant A as "API Gateway"
    participant O as "Order Service"
    participant P as "Payment Service (degraded)"
    U->>A: 500 req/s
    A->>O: forward
    O->>P: charge()
    Note over P: latency goes from 20 ms to 20 s
    Note over O: threads blocked waiting on P, pool exhausted
    A->>O: more requests
    O--xA: queue full, timeouts
    Note over A: gateway threads now blocked on O
    U--xA: site-wide errors, even for pages that never touch payments

Slow is worse than dead: a refused connection fails in a millisecond, but a hung one holds resources until the timeout fires, so most real cascades start with slowness. Retries make it worse (three layers that each try three times turn one request into up to 27 calls on the bottom service), and a restarted service often comes back to a wall of retries and falls over again.

A circuit breaker cuts this at the first hop. Once it decides a dependency is unhealthy, calls fail in microseconds instead of seconds, the caller's resources stay free, and the dependency gets a period with little or no traffic in which it can recover.

A Brief History

The term comes from Michael Nygard's book Release It! (Pragmatic Bookshelf, 2007), which described stability patterns drawn from production outages: timeouts, bulkheads, circuit breakers, fail fast, and steady state. Martin Fowler's short 2014 article "CircuitBreaker" spread the idea further.

Netflix turned the pattern into infrastructure. Hystrix, open-sourced in 2012, wrapped each dependency call in a "command" object that ran on its own thread pool (a bulkhead), enforced a timeout, tracked success and failure in a rolling ten-second window split into one-second buckets, opened the circuit when the error percentage crossed a threshold, and offered a fallback hook. Its real-time dashboard did a lot to popularise the pattern. In November 2018 Netflix announced that Hystrix was in maintenance mode: no new features, with users pointed to resilience4j for new work, while Netflix itself moved toward adaptive approaches such as its concurrency-limits library.

resilience4j started as a lightweight library designed for Java 8 functional programming. Instead of one heavy command abstraction it offers separate decorators (CircuitBreaker, Retry, RateLimiter, Bulkhead, TimeLimiter, Cache) that wrap any Supplier, Function or CompletionStage, and integrations for Spring Boot, Micrometer, and reactive libraries. It does not force a thread pool per dependency, and its circuit breaker supports both count-based and time-based sliding windows and a separate slow-call threshold.

Elsewhere: Polly in .NET (whose v8 rewrite moved to "resilience pipelines"), sony/gobreaker in Go, and Envoy in the proxy layer, which offers concurrency-cap "circuit breaking" plus per-host "outlier detection", both covered below.

The State Machine

A circuit breaker has three states:

CLOSED counts failures → 3 failures trip it OPEN (instant rejection, no timeouts) → HALF-OPEN sends one probe → success closes the circuit.

Closed State (Normal)

All requests pass through. The circuit breaker monitors success and failure rates. If the failure rate exceeds the configured threshold within a measurement window, the circuit opens.

Open State (Failing Fast)

All requests are immediately rejected without calling the downstream service. This serves two purposes: it protects the failing service from additional load, and it protects the calling service from wasting resources on calls that will fail. After a configured timeout, the circuit transitions to half-open.

Half-Open State (Testing)

A limited number of probe requests are allowed through to test if the downstream service has recovered. If they succeed, the circuit closes. If any fail, the circuit reopens and the timeout resets.

The Full Transition Diagram

stateDiagram-v2
    [*] --> Closed
    Closed --> Closed: success, or failure below threshold
    Closed --> Open: trip condition met
    Open --> Open: call rejected immediately
    Open --> HalfOpen: wait duration elapsed
    HalfOpen --> Closed: probes succeed
    HalfOpen --> Open: a probe fails or is slow
    HalfOpen --> HalfOpen: extra calls rejected while probes run

The same thing as a table, which is how most implementations are actually written:

Current stateEventNext stateSide effects
ClosedCall succeedsClosedRecord success in the window
ClosedCall fails, trip condition not metClosedRecord failure
ClosedCall fails, trip condition metOpenRecord open time, emit event, reject from now on
OpenCall arrives before wait duration endsOpenReject with "circuit open" error, run fallback
OpenWait duration ends (timer or next call)Half-openReset probe counters
Half-openProbe permit availableHalf-openLet the call through, count it as a probe
Half-openNo probe permits leftHalf-openReject like Open
Half-openEnough probes succeedClosedClear the window, emit event
Half-openProbe fails, or failure rate among probes too highOpenRestart the wait, possibly with a longer duration

Two details are often missed. First, Open can become Half-open on a timer or lazily on the next call. The lazy way (used by the Go code below, and resilience4j's default unless automaticTransitionFromOpenToHalfOpenEnabled is set) needs no background task, but an idle breaker shows "Open" in dashboards forever. Second, Half-open must hand out a fixed number of probe permits (resilience4j's permittedNumberOfCallsInHalfOpenState, default 10). If it lets everything through, a recovering service gets the full load at once. The simple Go and TypeScript implementations on this page let every request through in Half-open; the production TypeScript version later on fixes that.

Extra States in Real Libraries

Production libraries add operational states to the core three. resilience4j has DISABLED (always allow, record nothing), FORCED_OPEN (always reject, for planned maintenance or a known-bad deploy) and METRICS_ONLY (record but never trip, for rolling out a new breaker in shadow mode). Even if your library lacks them, a manual "force open" override is worth adding: during an incident it is a much faster mitigation than a config change and a rolling restart.

Core Implementation

package circuitbreaker

import (
	"errors"
	"sync"
	"time"
)

type State int

const (
	StateClosed   State = iota
	StateOpen
	StateHalfOpen
)

func (s State) String() string {
	switch s {
	case StateClosed:
		return "CLOSED"
	case StateOpen:
		return "OPEN"
	case StateHalfOpen:
		return "HALF_OPEN"
	default:
		return "UNKNOWN"
	}
}

var ErrCircuitOpen = errors.New("circuit breaker is open")

type CircuitBreaker struct {
	mu               sync.Mutex
	state            State
	failureCount     int
	successCount     int
	failureThreshold int
	successThreshold int       // Successes needed in half-open to close
	timeout          time.Duration
	lastFailureTime  time.Time
	onStateChange    func(from, to State)
}

type Config struct {
	FailureThreshold int
	SuccessThreshold int
	Timeout          time.Duration
	OnStateChange    func(from, to State)
}

func New(cfg Config) *CircuitBreaker {
	return &CircuitBreaker{
		state:            StateClosed,
		failureThreshold: cfg.FailureThreshold,
		successThreshold: cfg.SuccessThreshold,
		timeout:          cfg.Timeout,
		onStateChange:    cfg.OnStateChange,
	}
}

func (cb *CircuitBreaker) Execute(fn func() error) error {
	if !cb.allowRequest() {
		return ErrCircuitOpen
	}

	err := fn()

	cb.mu.Lock()
	defer cb.mu.Unlock()

	if err != nil {
		cb.onFailure()
	} else {
		cb.onSuccess()
	}

	return err
}

func (cb *CircuitBreaker) allowRequest() bool {
	cb.mu.Lock()
	defer cb.mu.Unlock()

	switch cb.state {
	case StateClosed:
		return true
	case StateOpen:
		if time.Since(cb.lastFailureTime) > cb.timeout {
			cb.transitionTo(StateHalfOpen)
			return true
		}
		return false
	case StateHalfOpen:
		return true
	default:
		return false
	}
}

func (cb *CircuitBreaker) onSuccess() {
	switch cb.state {
	case StateClosed:
		cb.failureCount = 0
	case StateHalfOpen:
		cb.successCount++
		if cb.successCount >= cb.successThreshold {
			cb.transitionTo(StateClosed)
		}
	}
}

func (cb *CircuitBreaker) onFailure() {
	switch cb.state {
	case StateClosed:
		cb.failureCount++
		if cb.failureCount >= cb.failureThreshold {
			cb.transitionTo(StateOpen)
		}
	case StateHalfOpen:
		cb.transitionTo(StateOpen)
	}
	cb.lastFailureTime = time.Now()
}

func (cb *CircuitBreaker) transitionTo(newState State) {
	if cb.state == newState {
		return
	}
	oldState := cb.state
	cb.state = newState
	cb.failureCount = 0
	cb.successCount = 0

	if cb.onStateChange != nil {
		cb.onStateChange(oldState, newState)
	}
}

Usage:

cb := circuitbreaker.New(circuitbreaker.Config{
	FailureThreshold: 5,      // Open after 5 failures
	SuccessThreshold: 3,      // Close after 3 successes in half-open
	Timeout:          30 * time.Second,
	OnStateChange: func(from, to circuitbreaker.State) {
		log.Printf("Circuit breaker: %s → %s", from, to)
		metrics.RecordStateChange(from.String(), to.String())
	},
})

err := cb.Execute(func() error {
	return paymentService.Charge(ctx, amount)
})

if errors.Is(err, circuitbreaker.ErrCircuitOpen) {
	// Fallback: queue for retry, return cached result, etc.
	return handleFallback(ctx, amount)
}

Failure Detection Strategies

Not all errors should trip the circuit breaker. A 400 Bad Request means the client sent garbage — that is not a service failure. A 503 Service Unavailable means the service is actually down.

Error Classification

from enum import Enum

class ErrorType(Enum):
    TRANSIENT = "transient"      # Should trip circuit breaker
    PERMANENT = "permanent"      # Should NOT trip circuit breaker
    TIMEOUT = "timeout"          # Should trip circuit breaker

def classify_error(status_code: int, error: Exception) -> ErrorType:
    """Classify HTTP errors for circuit breaker decisions."""
    if isinstance(error, TimeoutError):
        return ErrorType.TIMEOUT

    if status_code is None:
        return ErrorType.TRANSIENT  # Connection error

    if 400 <= status_code < 500:
        return ErrorType.PERMANENT  # Client error, not a service failure

    if status_code >= 500:
        return ErrorType.TRANSIENT  # Server error

    return ErrorType.PERMANENT

Sliding Window Failure Rate

Instead of counting raw failures, track the failure rate over a sliding window:

public class SlidingWindowCircuitBreaker {
    private final int windowSize;
    private final double failureRateThreshold;
    private final CircularBuffer<Boolean> results;

    public SlidingWindowCircuitBreaker(int windowSize,
                                       double failureRateThreshold) {
        this.windowSize = windowSize;
        this.failureRateThreshold = failureRateThreshold;
        this.results = new CircularBuffer<>(windowSize);
    }

    public void recordResult(boolean success) {
        results.add(success);

        if (results.size() >= windowSize) {
            long failures = results.stream()
                .filter(r -> !r)
                .count();
            double failureRate = (double) failures / results.size();

            if (failureRate >= failureRateThreshold) {
                transitionToOpen();
            }
        }
    }
}

The failure rate threshold is typically set between 50-80%. The window size determines how many samples you need before making a decision — too small and you get false positives from normal error spikes; too large and you react too slowly.

Trip Policies: Consecutive Failures vs Failure Rate

The core design decision is the trip condition: what evidence is enough to open the circuit? There are two main families, plus refinements.

Consecutive failures

Open after N failures in a row; any success resets the counter. This is what the Go implementation above does, and it is the default in sony/gobreaker (more than 5 consecutive failures).

It is simple and needs almost no memory, but it is fragile in both directions:

  • It misses partial outages. Suppose one of four backend replicas is broken, so 25% of calls fail. Successes keep arriving between the failures, so the count rarely reaches 5. The breaker never opens, and a quarter of your users get errors indefinitely.
  • It trips on bursts at low volume. Five failures in a row on a service that gets one call a minute might just be a network blip spread over five minutes.

With independent failures at probability , a run of happens with probability : for , , but for , which trips within seconds at 1,000 rps. Sensitivity depends entirely on ; there is no knob for "what failure rate is acceptable".

Failure rate over a window

Open when the fraction of failures in the recent window crosses a threshold, for example 50%, but only once the window holds at least a minimum number of calls. This is what Hystrix, resilience4j and Polly's advanced breaker do.

It handles partial outages that consecutive counting misses, and it states the intent directly: "we will not keep calling a dependency that fails more than half the time". The minimum-calls guard is essential. Without it, the first call after a restart, if it fails, gives a 100% failure rate.

Comparison

AspectConsecutive failuresFailure rate over window
State keptOne counterRing buffer or time buckets
Catches partial failure (say 30% errors)RarelyYes, if threshold is below 30%
Behaviour at low trafficNoisy, trips on burstsNeeds minimum-call guard, may never evaluate
Behaviour at high trafficTrips fast on hard downTrips fast, smooth on noise
Knob that maps to business intentNoYes ("tolerate up to X% errors")
Good default forSmall clients, low-volume calls, hard-down detectionMost server-to-server traffic

A useful hybrid trips on either N consecutive failures (fast detection of hard down) or a failure rate above X% (detection of partial failure). A third trigger, slow-call rate, is covered below.

Sliding Windows in Depth

A failure-rate breaker needs a window: the set of recent calls whose outcomes it looks at. The two kinds behave differently.

Count-based windows

Keep the outcomes of the last N calls in a ring buffer. The failure rate is failures among those N.

flowchart LR
    subgraph RB["Ring buffer, size 10"]
        S1["ok"] --> S2["ok"] --> S3["fail"] --> S4["ok"] --> S5["fail"]
        S5 --> S6["fail"] --> S7["ok"] --> S8["fail"] --> S9["fail"] --> S10["fail"]
    end
    RB --> R["6 of 10 failed = 60%, above 50% so OPEN"]

To make the update O(1), keep a running failure count: when a new outcome overwrites a slot, subtract the old slot's value and add the new one. resilience4j's count-based window works this way.

The weakness is that a count-based window has no idea of time. At 2 requests per minute a window of 100 covers almost an hour; at 10,000 rps it covers 10 milliseconds, short enough for one GC pause to trip it.

Time-based windows

Keep outcomes from the last T seconds. Storing every call would cost memory proportional to traffic, so implementations split the window into buckets. Hystrix used a 10-second window of ten 1-second buckets by default. resilience4j's time-based window keeps one partial aggregation per second for slidingWindowSize seconds. Each bucket holds totals (calls, failures, slow calls, total duration), and the window total is the sum of the live buckets. As time moves on, the oldest bucket is dropped and a fresh one starts.

flowchart LR
    B1["t-4s: 100 calls, 2 fail"] --> B2["t-3s: 110 calls, 1 fail"]
    B2 --> B3["t-2s: 95 calls, 40 fail"] --> B4["t-1s: 105 calls, 70 fail"]
    B4 --> B5["now: 60 calls, 50 fail"]
    B5 --> SUM["total 470 calls, 163 fail = 34.7%"]

Cost is bounded by the bucket count, not the traffic, at the price of one-bucket resolution.

Choosing the window

Use a time-based window (10 to 60 seconds) for steady or spiky traffic, because its meaning does not change with traffic level. Use a small count-based window, or a consecutive-failure trigger, for low-volume calls that would otherwise rarely fill a time window.

How fast does a time-based breaker react? Suppose the dependency goes from 0% to 100% failure in an instant, and the window is 10 seconds with a 50% threshold. The breaker trips when failures make up half the window, which takes about 5 seconds at constant traffic. If that is too slow, shorten the window or add a consecutive-failure trigger. A 5-second detection at 1,000 rps means 5,000 failed calls before protection kicks in.

Slow Calls Are Failures Too

A dependency that has slowed from 50 ms to 4 seconds, but still answers correctly, does not trip an error-rate breaker. Yet it is the classic way cascades start, as described above. Modern breakers therefore watch latency too.

resilience4j's slowCallDurationThreshold defines what counts as slow, and slowCallRateThreshold sets the percentage of slow calls at which the breaker opens, evaluated over the same window as failures. Note that its defaults are deliberately inert: the slow threshold defaults to 60 seconds and the rate to 100%, so slow-call detection does nothing until you configure it.

Put the slow-call threshold well above the dependency's normal p99 and below the call's timeout, and size it by resource cost: at 200 rps with a pool of 100, Little's Law says the pool runs out once latency passes seconds.

Library Implementations

Resilience4j (Java)

import io.github.resilience4j.circuitbreaker.CircuitBreaker;
import io.github.resilience4j.circuitbreaker.CircuitBreakerConfig;
import io.github.resilience4j.circuitbreaker.CircuitBreakerRegistry;

CircuitBreakerConfig config = CircuitBreakerConfig.custom()
    .failureRateThreshold(50)                     // 50% failure rate
    .slowCallRateThreshold(80)                     // 80% slow call rate
    .slowCallDurationThreshold(Duration.ofSeconds(2))
    .waitDurationInOpenState(Duration.ofSeconds(30))
    .permittedNumberOfCallsInHalfOpenState(5)
    .slidingWindowType(SlidingWindowType.COUNT_BASED)
    .slidingWindowSize(10)
    .minimumNumberOfCalls(5)
    .recordExceptions(IOException.class, TimeoutException.class)
    .ignoreExceptions(BusinessException.class)
    .build();

CircuitBreakerRegistry registry = CircuitBreakerRegistry.of(config);
CircuitBreaker cb = registry.circuitBreaker("paymentService");

// Decorate a function
Supplier<PaymentResult> decorated = CircuitBreaker
    .decorateSupplier(cb, () -> paymentService.charge(amount));

// Execute with fallback
Try<PaymentResult> result = Try.ofSupplier(decorated)
    .recover(CallNotPermittedException.class,
             e -> PaymentResult.queued());

Polly (.NET)

using Polly;
using Polly.CircuitBreaker;

var circuitBreakerPolicy = Policy
    .Handle<HttpRequestException>()
    .Or<TimeoutException>()
    .AdvancedCircuitBreakerAsync(
        failureThreshold: 0.5,           // 50% failure rate
        samplingDuration: TimeSpan.FromSeconds(10),
        minimumThroughput: 8,            // Min calls before evaluating
        durationOfBreak: TimeSpan.FromSeconds(30),
        onBreak: (exception, duration) =>
        {
            logger.LogWarning(
                "Circuit opened for {Duration}s due to {Exception}",
                duration.TotalSeconds, exception.Message);
        },
        onReset: () => logger.LogInformation("Circuit closed"),
        onHalfOpen: () => logger.LogInformation("Circuit half-open")
    );

// Execute
var result = await circuitBreakerPolicy.ExecuteAsync(async () =>
{
    return await httpClient.GetAsync("/api/payments");
});

TypeScript Implementation

interface CircuitBreakerOptions {
  failureThreshold: number;
  successThreshold: number;
  timeout: number;  // milliseconds
}

class CircuitBreaker {
  private state: 'CLOSED' | 'OPEN' | 'HALF_OPEN' = 'CLOSED';
  private failureCount = 0;
  private successCount = 0;
  private lastFailureTime = 0;

  constructor(private options: CircuitBreakerOptions) {}

  async execute<T>(fn: () => Promise<T>): Promise<T> {
    if (this.state === 'OPEN') {
      if (Date.now() - this.lastFailureTime > this.options.timeout) {
        this.state = 'HALF_OPEN';
        this.successCount = 0;
      } else {
        throw new Error('Circuit breaker is OPEN');
      }
    }

    try {
      const result = await fn();
      this.onSuccess();
      return result;
    } catch (error) {
      this.onFailure();
      throw error;
    }
  }

  private onSuccess(): void {
    if (this.state === 'HALF_OPEN') {
      this.successCount++;
      if (this.successCount >= this.options.successThreshold) {
        this.state = 'CLOSED';
        this.failureCount = 0;
      }
    } else {
      this.failureCount = 0;
    }
  }

  private onFailure(): void {
    this.failureCount++;
    this.lastFailureTime = Date.now();

    if (this.state === 'HALF_OPEN') {
      this.state = 'OPEN';
    } else if (this.failureCount >= this.options.failureThreshold) {
      this.state = 'OPEN';
    }
  }
}

A Production-Grade TypeScript Breaker

The short version above shows the idea. A breaker you would ship needs a few more things: a time-bucketed window, a minimum-calls guard, slow-call detection, a limited number of half-open probes, a way to say which errors count, and jitter on the open duration so that a fleet of instances does not probe in lockstep. Here is one in about 150 lines with no dependencies.

type State = "CLOSED" | "OPEN" | "HALF_OPEN";

export class CircuitOpenError extends Error {
  constructor(public readonly breaker: string, public readonly retryAfterMs: number) {
    super(`circuit '${breaker}' is open`);
  }
}

export interface BreakerOptions {
  name: string;
  windowSeconds: number;          // length of the time-based window
  minimumCalls: number;           // do not evaluate below this many calls
  failureRateThreshold: number;   // 0..1, e.g. 0.5
  slowCallMs: number;             // calls slower than this count as slow
  slowCallRateThreshold: number;  // 0..1
  openMs: number;                 // base wait in OPEN before probing
  maxOpenMs: number;              // cap for repeated trips
  halfOpenProbes: number;         // permits in HALF_OPEN
  isFailure?: (err: unknown) => boolean; // default: everything is a failure
  now?: () => number;             // injectable clock for tests
  onStateChange?: (from: State, to: State) => void;
}

interface Bucket { epochSec: number; calls: number; failures: number; slow: number; }

export class CircuitBreaker {
  private state: State = "CLOSED";
  private buckets: Bucket[];
  private openedAt = 0;
  private currentOpenMs: number;
  private consecutiveTrips = 0;
  private probesInFlight = 0;
  private probeSuccesses = 0;
  private readonly now: () => number;

  constructor(private readonly o: BreakerOptions) {
    this.now = o.now ?? Date.now;
    this.buckets = Array.from({ length: o.windowSeconds }, () => ({ epochSec: -1, calls: 0, failures: 0, slow: 0 }));
    this.currentOpenMs = o.openMs;
  }

  getState(): State {
    this.maybeHalfOpen();
    return this.state;
  }

  async execute<T>(fn: () => Promise<T>): Promise<T> {
    this.maybeHalfOpen();

    if (this.state === "OPEN") {
      throw new CircuitOpenError(this.o.name, this.openedAt + this.currentOpenMs - this.now());
    }
    const isProbe = this.state === "HALF_OPEN";
    if (isProbe) {
      if (this.probesInFlight + this.probeSuccesses >= this.o.halfOpenProbes) {
        throw new CircuitOpenError(this.o.name, 0); // probes already running
      }
      this.probesInFlight++;
    }

    const start = this.now();
    try {
      const result = await fn();
      this.record(false, this.now() - start, isProbe);
      return result;
    } catch (err) {
      const counts = this.o.isFailure ? this.o.isFailure(err) : true;
      this.record(counts, this.now() - start, isProbe);
      throw err;
    } finally {
      if (isProbe) this.probesInFlight--;
    }
  }

  private record(failed: boolean, durationMs: number, isProbe: boolean): void {
    const slow = durationMs > this.o.slowCallMs;

    if (isProbe && this.state === "HALF_OPEN") {
      if (failed || slow) {
        this.trip();             // one bad probe sends us back to OPEN
      } else if (++this.probeSuccesses >= this.o.halfOpenProbes) {
        this.close();
      }
      return;
    }
    if (this.state !== "CLOSED") return; // late result from an old call

    const b = this.bucketFor(this.now());
    b.calls++;
    if (failed) b.failures++;
    if (slow) b.slow++;
    this.evaluate();
  }

  private evaluate(): void {
    const nowSec = Math.floor(this.now() / 1000);
    let calls = 0, failures = 0, slow = 0;
    for (const b of this.buckets) {
      if (nowSec - b.epochSec < this.o.windowSeconds) {
        calls += b.calls; failures += b.failures; slow += b.slow;
      }
    }
    if (calls < this.o.minimumCalls) return;
    if (failures / calls >= this.o.failureRateThreshold || slow / calls >= this.o.slowCallRateThreshold) {
      this.trip();
    }
  }

  private bucketFor(nowMs: number): Bucket {
    const sec = Math.floor(nowMs / 1000);
    const b = this.buckets[sec % this.buckets.length];
    if (b.epochSec !== sec) { b.epochSec = sec; b.calls = 0; b.failures = 0; b.slow = 0; }
    return b;
  }

  private maybeHalfOpen(): void {
    if (this.state === "OPEN" && this.now() - this.openedAt >= this.currentOpenMs) {
      this.transition("HALF_OPEN");
      this.probesInFlight = 0;
      this.probeSuccesses = 0;
    }
  }

  private trip(): void {
    this.consecutiveTrips++;
    // Exponential growth for repeated trips, capped, with +/-20% jitter.
    const base = Math.min(this.o.openMs * 2 ** (this.consecutiveTrips - 1), this.o.maxOpenMs);
    this.currentOpenMs = base * (0.8 + Math.random() * 0.4);
    this.openedAt = this.now();
    this.transition("OPEN");
  }

  private close(): void {
    this.consecutiveTrips = 0;
    this.currentOpenMs = this.o.openMs;
    for (const b of this.buckets) { b.epochSec = -1; b.calls = 0; b.failures = 0; b.slow = 0; }
    this.transition("CLOSED");
  }

  private transition(to: State): void {
    if (to === this.state) return;
    const from = this.state;
    this.state = to;
    this.o.onStateChange?.(from, to);
  }
}

Late results from calls that started before a trip are ignored, so they cannot pollute a window that no longer matters. Repeated trips double the open period up to maxOpenMs, and the jitter stops a fleet that tripped together from probing together. The injectable clock makes it testable without sleeping.

A test that drives the state machine with a fake clock:

import assert from "node:assert/strict";

let t = 0;
const cb = new CircuitBreaker({
  name: "payments", windowSeconds: 10, minimumCalls: 4, failureRateThreshold: 0.5,
  slowCallMs: 1_000, slowCallRateThreshold: 1.1, openMs: 5_000, maxOpenMs: 60_000,
  halfOpenProbes: 2, now: () => t,
});
const ok = () => Promise.resolve("ok");
const boom = () => Promise.reject(new Error("503"));

await cb.execute(ok);
await cb.execute(ok);
await assert.rejects(cb.execute(boom));
await assert.rejects(cb.execute(boom));        // 2 of 4 = 50%, trips
assert.equal(cb.getState(), "OPEN");
await assert.rejects(cb.execute(ok), CircuitOpenError);

t += 7_000;                                     // past 5 s plus up to 20% jitter
assert.equal(cb.getState(), "HALF_OPEN");
await cb.execute(ok);
await cb.execute(ok);                           // 2 probes succeed
assert.equal(cb.getState(), "CLOSED");
console.log("breaker state machine ok");

Things this version leaves out on purpose: sharing state across processes (usually a bad idea, see below), and Promise cancellation (pair it with an AbortSignal timeout so the call itself is bounded).

Complementary Patterns

Circuit breakers rarely work alone. They are part of a resilience stack.

Retry with Exponential Backoff

Retry transient failures before the circuit breaker trips:

graph TD
    A["Request"] --> B["Retry Policy"]
    B --> C["Circuit Breaker"]
    C --> D["Timeout"]
    D --> E["Actual Call"]
    E -->|Success| F["Return Result"]
    E -->|Failure| B
    C -->|Open| G["Fallback"]
    B -->|Max retries| G
import time
import random

def retry_with_backoff(fn, max_retries=3, base_delay=1.0, max_delay=30.0):
    """Retry with exponential backoff and full jitter."""
    for attempt in range(max_retries + 1):
        try:
            return fn()
        except TransientError as e:
            if attempt == max_retries:
                raise

            delay = min(base_delay * (2 ** attempt), max_delay)
            jitter = random.uniform(0, delay)
            time.sleep(jitter)

Retry + Circuit Breaker Ordering

Always put retry outside the circuit breaker. With retry outside, every attempt passes through the breaker, so the breaker sees the real per-attempt failure rate, and once it opens, the remaining retries fail fast with "circuit open" instead of hitting the struggling dependency. If retry is inside, the breaker only sees the final result of each retried call, so it under-counts failures and reacts later, and nothing stops the retry loop from hammering the dependency in the meantime.

Correct: Retry → CircuitBreaker → Call Wrong: CircuitBreaker → Retry → Call

With retry outside, make sure the retry policy does not retry on the circuit-open error itself (or retries it only after the advertised wait). Otherwise a client spins in a tight loop against an open breaker.

Correction

An earlier version of this warning gave the reason the other way round: it said that retry inside the breaker makes each retry count as a separate failure. It is the outside placement in which every attempt is recorded, and that is the desired behaviour, not premature tripping. The recommended order was right. The explanation has been fixed. This is also the default aspect order in resilience4j's Spring Boot integration: Retry is outermost, then CircuitBreaker, then RateLimiter, TimeLimiter and Bulkhead around the call.

Bulkhead Pattern

Isolate failures by partitioning resources. If the payment service fails, it should not consume all your thread pool capacity and starve the order service.

graph TD
    subgraph "Without Bulkhead"
        A1["All Calls"] --> P1["Shared Thread Pool<br/>(100 threads)"]
        P1 --> S1["Payment Service (stuck)"]
        P1 --> S2["Order Service"]
        P1 --> S3["User Service"]
    end

    subgraph "With Bulkhead"
        A2["Payment Calls"] --> P2["Pool A<br/>(30 threads)"]
        A3["Order Calls"] --> P3["Pool B<br/>(40 threads)"]
        A4["User Calls"] --> P4["Pool C<br/>(30 threads)"]
        P2 --> S4["Payment Service (stuck)"]
        P3 --> S5["Order Service ✓"]
        P4 --> S6["User Service ✓"]
    end
// Resilience4j Bulkhead
BulkheadConfig config = BulkheadConfig.custom()
    .maxConcurrentCalls(30)
    .maxWaitDuration(Duration.ofMillis(500))
    .build();

Bulkhead paymentBulkhead = Bulkhead.of("payment", config);

// Combine with circuit breaker
Supplier<PaymentResult> decorated = Decorators
    .ofSupplier(() -> paymentService.charge(amount))
    .withBulkhead(paymentBulkhead)
    .withCircuitBreaker(circuitBreaker)
    .withRetry(retry)
    .withFallback(List.of(
        CallNotPermittedException.class,
        BulkheadFullException.class
    ), e -> PaymentResult.queued())
    .decorate();

Timeout Pattern

Always pair circuit breakers with timeouts. Without a timeout, a hanging call never returns an error, so the circuit breaker never sees a failure.

func callWithTimeout(ctx context.Context, timeout time.Duration,
                     fn func(context.Context) error) error {
    ctx, cancel := context.WithTimeout(ctx, timeout)
    defer cancel()

    done := make(chan error, 1)
    go func() {
        done <- fn(ctx)
    }()

    select {
    case err := <-done:
        return err
    case <-ctx.Done():
        return fmt.Errorf("call timed out after %v", timeout)
    }
}

How Retries, Timeouts and Breakers Compose

Each of the three handles a different failure shape, and each one breaks if it is used without the others:

MechanismHandlesFails without the others because
TimeoutHung or very slow callsAlone, it bounds one call, but every call still pays the full timeout during an outage
RetryShort, independent glitches (one dropped packet, one bad replica)Alone, it multiplies load during a real outage
Circuit breakerSustained failureWithout a timeout it never sees a hung call as a failure. Without retries, single glitches surface as errors.

The full picture of a single logical request, with an overall deadline:

sequenceDiagram
    participant C as "Caller"
    participant R as "Retry (max 3, backoff)"
    participant B as "Breaker"
    participant T as "Timeout 300 ms"
    participant D as "Dependency"
    C->>R: request, deadline 1 s
    R->>B: attempt 1
    B->>T: allowed (Closed)
    T->>D: call
    D--xT: no reply
    T-->>B: timeout, recorded as failure
    B-->>R: failure
    Note over R: wait 50 ms plus jitter
    R->>B: attempt 2
    B-->>R: circuit just opened, rejected in microseconds
    Note over R: do not retry circuit-open errors
    R-->>C: fail fast, caller runs fallback

Rules of thumb that keep the combination safe:

  1. Timeouts shrink as you go down. Pass a deadline down (gRPC does this natively) and set each attempt's timeout from the remaining budget. Three attempts of 300 ms plus two 100 ms backoffs is 1.1 s, which does not fit a 1 s deadline.
  2. Retry at one layer. Attempts multiply as for attempts per layer and layers.
  3. Use a retry budget, such as "retries may add at most 10% to the request rate", so retries cannot double load during an outage. Google's SRE book describes this per-client budget, and Envoy and gRPC support similar limits.
  4. Only retry what is safe. A timed-out POST /charge may have succeeded, so non-idempotent operations need idempotency keys. See Exactly-Once Semantics.

Retries, backoff and jitter have a page of their own: Timeouts, Retries and Backoff.

Fallback Strategies

When the circuit is open, you need a fallback. The right fallback depends on your business requirements:

StrategyWhen to UseExample
Default valueNon-critical dataReturn empty recommendations
CacheStale data is better than no dataServe cached product prices
QueueEventually consistent is OKQueue payment for retry
Graceful degradationPartial service is better than noneShow product page without reviews
Error responseClient can handle itReturn 503 with retry-after
class ResilientProductService:
    def __init__(self, product_api, cache, circuit_breaker):
        self.api = product_api
        self.cache = cache
        self.cb = circuit_breaker

    def get_product(self, product_id: str) -> Product:
        try:
            product = self.cb.execute(
                lambda: self.api.get(product_id)
            )
            self.cache.set(product_id, product, ttl=3600)
            return product
        except CircuitOpenError:
            # Fallback 1: serve from cache
            cached = self.cache.get(product_id)
            if cached:
                cached.stale = True
                return cached
            # Fallback 2: minimal response
            return Product(id=product_id, name="Temporarily unavailable",
                          available=False)

Fallback Pitfalls

A fallback is code that only runs during incidents, which makes it the least tested code in the system. Common ways it goes wrong:

  • The fallback shares the failure. The fallback for "pricing service down" reads prices from the same database the pricing service was choking on. Now the database gets the pricing service's load plus the fallback's. A fallback must use a different resource, or a much cheaper path to the same one.
  • The fallback is more expensive than the call. "If recommendations fail, compute a simple ranking from the full catalogue" can be heavier than the call it replaces, and it runs on every request while the breaker is open.
  • The fallback lies. Returning available: true from a stale cache for an item that sold out, or "payment accepted" when the charge was only queued, turns an outage into a correctness bug. The page above marks cached products as stale. Callers need that flag to decide what to show.
  • The fallback hides the outage. If every breaker has a silent fallback, dashboards show 100% success while customers get empty carts. Count fallback invocations as their own metric and alert on them. Sort each dependency into critical with no substitute (payment authorisation: fail clearly or queue), stale-is-OK (prices, profiles: serve cached values with a staleness marker) or optional (recommendations, reviews: omit and count). The test for a fallback: run a game day where the breaker is forced open (the forced-open state from earlier) and watch what users actually see.

Monitoring Circuit Breakers

Circuit breaker state changes are critical operational events. Monitor them aggressively.

from prometheus_client import Counter, Gauge, Histogram

# Metrics
circuit_state = Gauge(
    'circuit_breaker_state',
    'Current state (0=closed, 1=open, 2=half-open)',
    ['service']
)
circuit_transitions = Counter(
    'circuit_breaker_transitions_total',
    'State transitions',
    ['service', 'from_state', 'to_state']
)
call_duration = Histogram(
    'circuit_breaker_call_duration_seconds',
    'Call duration through circuit breaker',
    ['service', 'result']
)

Alert on Open Circuits

Set up alerts for circuit breaker state changes. An open circuit means a dependency is failing, and you need to investigate. Also track the time spent in open state — if a circuit stays open for more than a few minutes, something is seriously wrong.

See Observability for alerting strategies.

Anti-Patterns

Circuit Breaker Per Instance vs Per Service

If you have a circuit breaker per downstream instance and one instance dies, the circuit breaker for that instance opens, but all other instances keep receiving traffic. This is correct. If you have a single circuit breaker for the entire service, one bad instance causes you to stop calling all instances — even the healthy ones.

Too-Sensitive Thresholds

Setting failureThreshold = 1 causes the circuit to open on a single failure. Network blips happen. Set thresholds high enough to filter out noise but low enough to catch real outages. Start with a failure rate of 50% over a window of 10 calls.

No Monitoring

A circuit breaker silently failing fast is worse than a visible error. If you cannot tell which circuits are open and why, you have traded one problem for another.

Per-Dependency Breakers and Granularity

"One breaker per dependency" sounds obvious, but "dependency" can be cut at several levels, and the choice decides what one failure can take down.

GranularityExample keyIsolatesRisk
Per servicepaymentsOne service from anotherOne bad endpoint or replica blocks all of payments
Per endpointpayments:POST /chargeExpensive or fragile operations from cheap onesMore breakers to configure and watch
Per host or replicapayments@10.0.3.17A bad replica from good onesNeeds the load balancer to route around open hosts
Per tenant or shardsearch:shard-12One hot or broken partition from the restMany breakers, sparse traffic per breaker

Common good defaults:

  • Per endpoint (or per operation class) inside the app. GET /profile and POST /export-report on the same service fail in different ways and at different rates. One breaker for both lets the expensive one trip the cheap one.
  • Per host at the load-balancer or proxy layer. This is what Envoy's outlier detection does. By default, a host that returns 5 consecutive 5xx responses is ejected from the pool for a base ejection time of 30 seconds, with the time growing with each repeated ejection. A cap (max_ejection_percent, 10% by default) stops it from ejecting so many hosts that the rest are overloaded.
  • Per shard for partitioned data stores. If shard 12's primary is down, reads to shard 12 should fail fast while the other shards keep working. A single "database" breaker would turn a one-shard outage into a total one.

Keep breakers in a registry (a map from key to breaker, created on first use, like resilience4j's CircuitBreakerRegistry), and watch the cardinality. Keying breakers by user ID or by full URL with path parameters creates millions of breakers, each with too little traffic to ever reach its minimum calls, and leaks memory.

Envoy's two mechanisms

Envoy uses the word "circuit breaking" for something different from the application pattern on this page, which confuses people:

Envoy featureWhat it actually doesDefaults per cluster
Circuit breakers (circuit_breakers.thresholds)Concurrency caps. When the limit is hit, new requests fail immediately with 503 and an overflow counter goes up.max_connections 1024, max_pending_requests 1024, max_requests 1024, max_retries 3
Outlier detectionPer-host ejection based on consecutive errors, success rate, or failure percentageconsecutive_5xx 5, base_ejection_time 30 s, max_ejection_percent 10

The concurrency cap is really a bulkhead: it bounds how much of the proxy's capacity one upstream can use. Outlier detection is the per-host breaker. Neither one trips on the overall error rate of a whole cluster. That is still the application's job, or a job for retry budgets and load shedding.

Do not share breaker state across instances

It is tempting to store breaker state in Redis so the whole fleet trips together. Usually this is a mistake. It puts a new shared dependency on the hot path, and it throws away per-instance judgement: one zone's network path may be broken while another's is fine. Local breakers across a fleet already trip at about the same time when a dependency is truly down.

Tuning Guide

Every number in a breaker config is a trade-off between reacting quickly and not reacting to noise. A worked tuning pass for a typical dependency:

The dependency: a pricing service, 400 rps from each caller instance, normal error rate 0.2%, p50 25 ms, p99 180 ms. Callers have a 200-thread pool and a 1-second end-user deadline.

KnobChoiceReasoning
Window typeTime-basedTraffic is steady and high, so a time window has a stable meaning
Window length10 sAbout 4,000 calls per window, so the rate is statistically solid. Detection of a hard failure takes about 5 s at a 50% threshold.
Minimum calls50Guards startup and quiet periods. Even off-peak at 20 rps, the window holds 200 calls.
Failure-rate threshold25%Normal is 0.2%. At 25%, one in four users is already getting errors, so we would rather serve cached prices.
Slow-call threshold500 msWell above p99 (180 ms) and below the 1 s deadline. At 400 rps and 500 ms, 200 calls would be in flight, which is the entire pool.
Slow-call rate50%Trips when half of all calls are slow enough to threaten the pool
Wait in Open10 s, jittered, doubling to 60 sLong enough for a restart or failover, short enough that recovery is noticed quickly
Half-open probes5Enough to tell recovery from one lucky call, not enough to dump load on a recovering service
Errors that count5xx, timeouts, connection errors4xx is the caller's problem and should not trip anything

With a normal error probability of 0.2%, the binomial chance that 13 of 50 calls fail independently is negligible. False trips come from correlated errors, such as a 2-second GC pause on the dependency, which is why the window should hold several seconds of history.

Library defaults to know

LibraryDefault trip ruleDefault windowDefault waitHalf-open
Hystrix50% errors, at least 20 requests10 s rolling, 10 buckets5 s sleep windowOne trial request
resilience4j50% failures, at least 100 callsCount-based, 100 calls60 s10 permitted calls
gobreakerMore than 5 consecutive failuresCounts cleared every Interval (0 means never in Closed)60 sMaxRequests, default 1
Envoy outlier detection5 consecutive 5xxPer host30 s base ejection, growingHost returns after ejection time

These defaults are starting points chosen by the library authors, not recommendations for your traffic. resilience4j's 100-call minimum in particular means a low-traffic dependency may never trip at all.

Failure Scenarios

Flapping

The breaker opens, probes succeed, it closes, the full load returns, the dependency falls over again, and the cycle repeats every 30 seconds. The dependency can handle a trickle but not full load. Fixes: more and longer probing in Half-open, ramping traffic up after closing instead of switching from 0% to 100% at once (a "slow start", which Envoy supports for load balancing), and backing off the open duration on repeated trips as the TypeScript breaker does.

Thundering herd on close

Five hundred instances tripped at the same time during the outage, and with identical wait durations they all probe at the same time and all close within the same second. The dependency goes from zero to full traffic plus a backlog of client retries. Jittering the wait duration spreads the probes. Ramping after close spreads the load.

Partial failure that never trips

One of eight replicas is broken, so 12.5% of calls fail. A service-level breaker with a 50% threshold never opens, and one user in eight keeps seeing errors. This is the case for per-host breaking (outlier detection) at the load-balancer level, plus retrying on a different host.

The breaker trips on the caller's own bug

A deploy introduces a bug that sends malformed requests, the dependency returns 500 (when it should have returned 400), and the breaker opens, cutting off all traffic, including requests from the instances still on the old version. Classify errors carefully, and watch for breakers that open right after your own deploy rather than the dependency's.

Low-traffic breakers that never trip

A nightly batch job calls a dependency 30 times. With a minimum of 100 calls the breaker never evaluates, and the job grinds through 30 full timeouts. For low-volume calls, use a consecutive-failure trigger or a small count-based window.

War Story (illustrative)

A retail site had a breaker on its inventory service: 50% failure threshold, 30-second wait, unlimited calls in Half-open. During a database failover, inventory went down for about 90 seconds. The breaker opened as designed. Thirty seconds later, every one of about 300 checkout instances moved to Half-open within the same second, because they had all tripped together. The recovering inventory service got the full checkout load, plus the retry backlog from mobile clients, in one burst. It fell over again, and the breakers reopened. This repeated four times, and a 90-second database failover turned into a 9-minute checkout outage. The fix was three lines of configuration: jitter the wait (plus or minus 25%), allow only 5 calls in Half-open, and double the wait on each consecutive trip. At the next failover, checkout recovered within 20 seconds of the database.

Beyond Fixed Thresholds

A breaker with fixed thresholds is a binary switch: all traffic or none. Some alternatives degrade more smoothly.

Client-side adaptive throttling. Google's SRE book (chapter "Handling Overload") describes clients that track, over the last two minutes, how many requests they sent and how many the backend accepted, and reject locally with probability

with typically 2. When the backend accepts everything, the probability is 0. As the backend rejects more, the client throttles more, in proportion. There is no Open state and no probing. A small trickle always gets through, and that trickle acts as a continuous probe.

Adaptive concurrency limits. Instead of tripping on errors, learn how much concurrency the dependency can take and cap in-flight requests there. Netflix's concurrency-limits library applies TCP-style congestion control (AIMD, Vegas-style latency gradients) to request concurrency, and Envoy has an adaptive concurrency filter based on the same ideas. When latency rises, the limit shrinks, and excess requests are rejected at once, which works like a breaker that opens partially.

These combine well with server-side load shedding on the dependency itself; see Queueing Theory for why a server near saturation must shed early.

Interview Questions

1. Walk through the three states of a circuit breaker and what triggers each transition.

Closed passes calls and records outcomes. When the trip condition is met (failure rate over a window, consecutive failures, or slow-call rate), it moves to Open, which rejects calls immediately. After a wait duration it moves to Half-open, which lets a limited number of probes through. If they succeed it closes. If any fail it reopens, often with a longer wait.

2. Why do you need a timeout if you have a circuit breaker?

The breaker only learns from completed calls. A call that hangs forever never completes, so it is never recorded as a failure, and the breaker never opens while threads pile up. The timeout turns "hung" into "failed" in bounded time.

3. Should retries go inside or outside the breaker?

Outside. Each attempt is then recorded by the breaker, and once it opens, remaining retries fail fast instead of hitting the dependency. The retry policy must not retry circuit-open errors. Retries inside the breaker hide failures from it and keep hammering the dependency.

4. Your breaker uses "5 consecutive failures" and one of four replicas is broken. What happens?

About 25% of calls fail, but successes keep resetting the counter, so the breaker rarely if ever opens and users keep seeing errors. Use a failure-rate policy, per-host outlier detection, or retry against another replica.

5. The breaker keeps flapping between Open and Closed. Why, and how do you fix it?

The dependency handles probe-level load but not full load. Fixes: more probes in Half-open, ramping traffic after closing, backing off the open duration on repeated trips, and jitter so instances do not move in lockstep.

6. How is Envoy's circuit breaking different from the Hystrix-style pattern?

Envoy's circuit breakers are concurrency caps per upstream cluster, which makes them a bulkhead. Per-host failure-based ejection is called outlier detection.

7. When would you choose adaptive throttling or a concurrency limiter over a classic breaker?

When the dependency is overloaded but still partly working, an on/off breaker either sends everything or nothing. Adaptive throttling rejects in proportion to the backend's rejection rate. A concurrency limiter caps in-flight work based on observed latency. Both degrade smoothly and keep a small probe stream going.

Further Reading

  • Rate Limiting — Complementary pattern for traffic control
  • Distributed Locking — Another pattern affected by failure modes
  • Failure Detectors — Theory behind detecting failed nodes
  • Consistency Models — How circuit breakers affect consistency guarantees
  • Service Mesh — How service meshes implement circuit breaking at the infrastructure level
  • Michael Nygard, Release It! — The book that popularized the circuit breaker pattern
  • Netflix Hystrix Wiki — The canonical implementation (now in maintenance mode)
  • Resilience4j documentation — The modern Java resilience library