Have you ever had one minor feature in your application, say a product recommendation module or a courier integration, suddenly go slow, and the impact took the whole application down? The checkout and login pages fell with it. 😅
In a microservice architecture, this terrifying scenario is called cascading failure, the domino effect of failure.
When one downstream service responds slowly, the upstream service keeps waiting on the connection until the timeout limit. As a result, hundreds of threads and upstream server connections get locked, the pool runs out of slots, and eventually the upstream falls too, following the troubled service.
To protect a system from this chain disaster, software architects install a legendary pattern called the circuit breaker pattern. Let's take apart how it saves your backend.
Key takeaways
- Cascading failure happens when slowness in one downstream service locks up the thread pool and brings the whole backend architecture down.
- A circuit breaker runs a three-state cycle: Closed (normal), Open (immediate fast-fail), and Half-Open (a gradual probe).
- A fallback strategy provides graceful degradation so users can still complete a transaction.
- The pattern protects server resources from hanging connections while a third-party service is down.
Why Is It Called a Circuit Breaker?
The term circuit breaker comes straight from the world of electrical wiring. If a house has a short circuit at the fridge outlet, what happens?
Without a fuse, the excess current keeps flowing. The cable inside the wall heats up, burns, and eventually the whole electrical installation in the house is destroyed.
With a fuse, the moment the sensor detects an abnormal current surge, its lever trips and cuts the flow to the kitchen area. The lights in the living room and bedroom stay on. The house is safe, it does not burn with it. 😄
In backend work, a circuit breaker acts exactly like that automatic safety switch. It cuts calls to the troubled service before that service damages the whole ecosystem.
The pattern was popularized by Martin Fowler's architecture article and is used in many industry libraries like Resilience4j and Netflix Hystrix.
The Three Main States of a Circuit Breaker
A circuit breaker works like a state machine with three main conditions.
Closed State: Everything Is Calm
In the normal condition, the switch is closed and the flow runs freely. Every request from the client passes straight to the target service. If an occasional error appears, say 1 in 100 requests fails, the circuit breaker only records the failure metric without cutting the flow.
Open State: Immediate Fast-Fail
If the failure or timeout percentage crosses the tolerance limit, for example 50% of requests failing in the last 10 seconds, the switch jumps straight to Open.
In this state, calls to the target service are cut. Every following request fails immediately (fast-fail), with no need to wait for a 10-30 second network timeout.
The system immediately runs a fallback mechanism: return data from an old cache, or show a friendly informational message. The troubled service also gets room to recover, instead of being bombarded by thousands of new requests.
Half-Open State: Probing Slowly
Once the rest period ends, say 30 seconds in the Open state, the switch moves to Half-Open.
Here the system allows a small number of probe requests (canary requests) to call the target service again. If the probe succeeds, the service is considered recovered and the switch returns to Closed. But if it still fails, it goes straight back to Open and resets the rest timer.
Fallback: Do Not Serve a Frightening Error Message
The key to this pattern's success is graceful degradation through a fallback response. A scary error message only makes users panic and leave.
If the recommendation service is down, show a default product list or a static best-seller list, rather than wrecking the layout of the home page.
If the courier or shipping rate service is down, show a temporary flat rate estimate, with a note that the detailed calculation will be updated at checkout.
If the rating and review service is down, hide the review section for now. Users can still read the product description and check out calmly.
An Implementation Example in Python (FastAPI + PyBreaker)
Here is an example of applying a circuit breaker with the PyBreaker and HTTPX libraries in Python:
import httpx
import pybreaker
from fastapi import FastAPI, HTTPException
from starlette.concurrency import run_in_threadpool
app = FastAPI()
# Circuit Breaker configuration:
# Trip to Open if 5 consecutive requests fail, retry after 30 seconds (reset_timeout)
payment_breaker = pybreaker.CircuitBreaker(
fail_max=5,
reset_timeout=30,
name="PaymentGatewayBreaker"
)
def call_payment_gateway(order_id: str, amount: float):
# Call the third-party payment gateway endpoint (synchronous, as pybreaker expects)
with httpx.Client(timeout=3.0) as client:
response = client.post(
"https://api.external-payment.com/v1/charge",
json={"order_id": order_id, "amount": amount}
)
response.raise_for_status()
return response.json()
@app.post("/api/checkout/{order_id}")
async def checkout(order_id: str, amount: float):
try:
# Run the call inside the Circuit Breaker protection.
# pybreaker.call() is synchronous, so we push it to the threadpool
# so the FastAPI event loop does not block.
result = await run_in_threadpool(
payment_breaker.call, call_payment_gateway, order_id, amount
)
return {"status": "success", "data": result}
except pybreaker.CircuitBreakerError:
# The switch is OPEN (Fast-Fail): Do not wait for a timeout, fall back immediately!
return {
"status": "degraded",
"message": "Gerbang pembayaran sedang dalam pemeliharaan berkala. Pesanan kalian sudah dicatat dan akan diproses otomatis begitu sistem pulih!",
"order_id": order_id
}
except Exception as exc:
raise HTTPException(
status_code=502,
detail=f"Terjadi kendala pada mitra pembayaran: {str(exc)}"
)
With that pattern, when the third-party payment server is completely down, your checkout API will not hang or run out of connections. The client gets an informative response within a few milliseconds.
So, Why Is This Mandatory?
In a complex distributed architecture, failure of a third-party component or a supporting service is inevitable (failures are inevitable).
Instead of letting a single point of failure cripple the whole application, installing the circuit breaker pattern is mandatory defense. Your core system stays on, resilient, and elegant when crisis arrives.
Happy architecting, and may one collapsing service never bring the rest down. 👋