Circuit Breaker
Track: Load and timing. 6 stages. Languages: Go, Python, JavaScript, Rust.
When a service you depend on starts failing, calling it again and again only makes things worse. You pile more load onto something that is already struggling, and you tie up your own threads waiting for answers that never come. A circuit breaker watches the calls going through it: once a service is clearly broken, it stops sending calls its way for a while, then lets one careful test call through to check whether it has recovered, before trusting it with everything again.
Your breaker moves through three states:
- Closed: calls pass through as normal. Failures in a row are counted, and a success resets that count back to zero.
- Open: calls are rejected right away, with no real attempt made. After a set wait, the very next call is let through as a test instead of being rejected.
- Half-open: only a small number of test calls are let through. Enough of them succeeding closes the breaker again. Even one of them failing sends it straight back to open, and the wait starts over.
Here is one example timeline, with a failure limit of 3, a wait of 60 ms, and 2 test calls allowed: call 1 fails, call 2 fails, call 3 fails. That is 3 in a row, so the breaker trips to open. Calls 4 and 5 are rejected right away. 60 ms pass. Call 6 is let through as a test (the breaker is half-open) and it succeeds. Call 7 is also let through as a test and it succeeds too, which makes 2 successful test calls, so the breaker closes. Call 8 passes through as normal again.
Implement New(failureThreshold, recoveryTimeout, halfOpenMax), Allow() -> bool, ReportSuccess(), ReportFailure(), and State() -> closed|open|half_open using your language's own names. Call Allow before every real attempt. If it says yes, make the real call, then report whether it succeeded or failed. Never wait or sleep inside Allow, decide purely from the clock and the state you are already holding.
Your breaker will be called from many callers at once, so it must stay correct when several calls happen together. Standard library only.
The tester calls your breaker in real time, with real waits, and writes down exactly when each call happened and what your breaker decided. It then checks those decisions against your own timing, so small timing differences caused by your machine being busy or slow can never fail you. Only a wrong decision, such as letting a call through when it should still be rejected, can fail you. For help with the ideas, read the notes.
Every stage below builds the breaker with a failure limit of 3, a wait of 60 ms, and 2 test calls allowed. Whenever a stage waits, it waits 140 ms, more than twice the 60 ms wait, so the real wait is safely crossed even if your machine is briefly slow.
Where it is used
- A shop's checkout calls a payment service. When that service hangs, a breaker makes checkout fail fast with a clear message, instead of every request waiting for a timeout and using up the shop's own threads.
- Netflix built Hystrix, and the Java library Resilience4j does the same job today: they wrap calls to other services in a breaker, so one slow service does not take down every service that calls it.
- A breaker that never lets a test call through stays open forever, even after the service has recovered. A breaker that lets everything through at once can knock a service over again just as it comes back.
Stages
cb1: hello, a closed breaker lets every call through (very easy)
Create a fresh breaker and make 3 calls without reporting any outcome in between. All 3 must be let through, and the breaker must still report closed afterward. Example: New(3, 60, 2), then call Allow() three times with no ReportSuccess/ReportFailure in between -> all three return true, and State() is closed throughout.
The tester will verify: all 3 calls are allowed, and the breaker reports closed the whole time.
cb2: it trips after enough failures in a row (easy)
Make 3 calls that each fail, one after another. That is the failure limit, so the breaker must flip to open on the 3rd failure, and the 3 calls right after it must all be rejected. A success in the middle of a streak resets the count back to zero, so only a true run of failures with no success in between can trip the breaker.
The tester will verify: the breaker opens on the 3rd failure in a row and rejects the calls right after it, and a success partway through a streak resets the count instead of adding to it.
cb3: one test call after the wait (medium)
Trip the breaker with 3 failures in a row, then make 3 more calls while it is still open, which must all be rejected. Wait 30 ms and make 1 more call, which must be rejected too. Wait 40 ms more. That is 70 ms since the breaker opened, so the 60 ms wait is over. The wait counts from when the breaker opened. A rejected call does not start it again. Then make 4 more calls. Exactly 2 of those 4 must be let through, as test calls, with the breaker reporting half-open while it lets them through.
The tester will verify: no call is let through before the wait is over, then exactly 2 calls are let through as tests afterward, with the breaker reporting half-open.
cb4: the test calls decide what happens next (medium)
Two separate timelines, each starting the same way: trip the breaker, then wait past the 60 ms wait. In the first, both test calls succeed, so the breaker must close and let the 2 calls after that through as normal. In the second, the one test call that gets a chance to run fails, so the breaker must go straight back to open and reject the 2 calls after that.
The tester will verify: 2 successful test calls close the breaker, and a single failing test call reopens it, each checked by the calls that follow.
cb5: stress, failing and recovering again and again (hard)
One long run: trip (3 failures), wait, recover (2 successful tests), run normally for a bit, trip again (3 failures), wait, fail the one test call so it reopens, wait again, recover (2 successful tests), then run normally again.
The tester will verify: every call your breaker allows or rejects, and every state it reports, matches our solution exactly across the whole run.
cb6: allow and report speed (L2, hard)
Keep Allow and the two report calls fast, even under heavy use. Every answer must still stay correct. Note that cb1 to cb5 are timed too, so going slow in the everyday case can cost you those earlier stages as well, not just this one. Example: if our solution takes 100 ms, the limit is 150 ms. Your solution takes 140 ms, so it passes.
The tester will verify: your Allow and report calls take at most 1.5 times as long as our solution in the same language, and every answer stays correct. How many memory allocations you use is also shown, but it never decides a pass or a fail.
Rules
- Standard library only. No third-party packages.
- Never wait or sleep inside
Allow. Decide purely from the clock and the state you already hold. - Your breaker will be called from many callers at once. Keep every state change, including the clock reading you use to decide, safe under that.