Many callers at once
Why this page exists
Some challenges call your code from one caller at a time. Others call it from many callers at the same moment: the rate limiter, retries, the circuit breaker, idempotency keys, inventory, the ledger, the connection pool and the task queue. Code that is correct for one caller can be wrong for many. This page explains the one bug behind most of those failures and the usual fix.
What a race is
A race is a bug where the result depends on which caller gets there first. Each caller does something correct on its own. The problem is the order in which their steps mix together. Most runs are fine, because the steps happen not to mix in the bad way. Then one run mixes them badly, and the answer is wrong.
Check, then act
The most common race has a name: check-then-act. The code first checks something, then acts on what it saw. Between the check and the act, another caller can change the thing that was checked.
Here is a store with 1 unit left. Two buyers call Reserve for 1 unit at the same moment. Each Reserve reads the free count, checks that it is big enough, then writes the new count.
| Step | Buyer A | Buyer B | Free count |
|---|---|---|---|
| 1 | reads the free count: 1 | 1 | |
| 2 | reads the free count: 1 | 1 | |
| 3 | 1 is enough, writes 0, says yes | 0 | |
| 4 | 1 is enough, writes 0, says yes | 0 |
Both buyers were told yes. One unit was sold twice. Each buyer's code did the right thing with what it saw. Buyer B just saw a number that was already old.
Lost updates
A close cousin is the lost update. Two callers each add 1 to a counter that holds 5.
| Step | Caller A | Caller B | Counter |
|---|---|---|---|
| 1 | reads 5 | 5 | |
| 2 | reads 5 | 5 | |
| 3 | writes 6 | 6 | |
| 4 | writes 6 | 6 |
Two adds happened, but the counter went up by one. The work of caller A was written over and lost. In a ledger, this is money that disappears. In a rate limiter, it is a request that was let through but never counted.
How a lock fixes it
A lock lets only one caller at a time into a piece of code. A caller takes the lock, does its work, and gives the lock back. Any other caller that wants the lock waits until it is free.
The fix for both bugs is the same: take one lock, then do the check and the change, then give the lock back. With the lock, buyer B cannot read the free count until buyer A has written 0. So B reads 0 and says no. The check and the change now happen as one step that nothing can get in the middle of.
The lock must cover the whole read, check and write. A lock around the read only, and another one around the write only, does not help. The gap between them is still there.
What a lock costs
- Waiting. While one caller holds the lock, every other caller that needs it waits. Hold the lock for as short a time as you can.
- No slow work inside. Do not sleep, wait for the network or call code you do not control while you hold a lock. Every other caller waits for that too. The idempotency challenge shows the right shape: mark the key as "in progress" under the lock, run the work without the lock, then take the lock again to save the result.
- Deadlock. If caller A holds lock 1 and waits for lock 2, while caller B holds lock 2 and waits for lock 1, both wait forever. When you need two locks, always take them in the same order.
Two common ways to wait less:
- A lock per key. If calls for different keys never touch the same data, give each key its own lock. Calls for key "a" then never wait for calls for key "b".
- Compare and set. Some languages offer a single step that says "change this value from 5 to 6, but only if it is still 5". If another caller got there first, the step fails and you read again and try again. This works well for one number, such as a counter.
In each language
The tools have different names, but the idea is the same in every language.
Go. Use sync.Mutex. Call mu.Lock() before the check and defer mu.Unlock() right after it, so the lock is given back even if the function returns early. Each goroutine is a caller, and they can truly run at the same time.
Python. Use threading.Lock. Write with self.lock: and put the check and the change inside that block. The lock is given back when the block ends. Python's global lock does not protect you here. A thread can be paused between any two lines of your code, so check-then-act still races.
Rust. Put the data inside a std::sync::Mutex. Calling .lock() gives you a guard, and you reach the data through that guard. The lock is given back when the guard goes out of scope. Rust will not let you touch the data without the lock, which makes this bug hard to write. It is still possible if you take the lock to read, let the guard go, and take it again to write.
JavaScript. Only one piece of your code runs at a time, so a check and a change with no await between them cannot be split. The race happens across an await. If your code reads a value, waits for something with await, and then writes, other calls run during that wait and can read the same old value. In the challenges that give you an async db, every get and set is such a wait. The fix is a lock per key built from a chain of promises: each call for a key waits for the call before it to finish, then runs, then lets the next one go. Keep a map from key to the last promise in its chain.
How Faultline shows it
A race may not show up on any single run. So the stages that test many callers do two things:
- A storm. Many callers hit your code at the same moment, for example 16 buyers for the last 5 units. In Go, Python and Rust they run on real threads. In JavaScript they are async calls that overlap at every
await. - Many rounds. The storm runs again and again, often 100 times, each round from a fresh start. Every round must give exactly the right answer.
Each stage is also run a few times. If it passed on some runs and failed on others, the result is FLAKY. That means "passed sometimes", which is what a race looks like. Do not run the check again and hope for a pass. Look for a place where your code reads a value, decides, and then writes, and make sure one lock covers all three steps.
A slow or busy machine can make a race show up more often or less often. It does not create a race in code that has none. So if you see FLAKY on a stage with many callers, look for a race first. On stages that measure time, such as the rate limiter, a guess about how long something takes can also pass sometimes and fail sometimes.