Idempotency Keys
What it is
A caller may retry a request when its response is lost, for example after a timeout. Repeating the work can charge a card twice or send a message twice. An idempotency key names one piece of work: the caller tags a request with a key, and the server guarantees that every repeat request with that key gets back the same result, without doing the work again.
The core idea
The first request for a key runs the work and saves its result. A later request with the same key reuses that saved result instead of running the work again.
A saved result does not last forever. It is kept fresh for a set amount of time, often called a time-to-live (TTL), and the cache only holds so many results at once. Once a result goes stale, or it gets pushed out to make room for newer ones, the next request with that key runs the work again and saves a new result.
Many copies of the same request can also arrive at the same instant, from different callers. Only one of them should actually run the work. The others must wait for that one call and then share its result, instead of each thinking the work hasn't started yet and running it themselves. Developers sometimes call this single-flighting a request.
Example: reusing a saved result
A request with key payment-7 starts a charge and gets back 42 as its result. A second request with the same key, moments later, returns 42 too, without charging the card again.
Now say results are kept fresh for 50 seconds. A request with key payment-9 at the 0-second mark charges the card and gets 7. A repeat at the 10-second mark is still inside the 50 seconds, so it also gets 7, with no new charge. A repeat at the 60-second mark is now past the 50 seconds, so the card is charged again, and this new request gets a new result, say 8.
Example: many callers at once
Say 10 copies of a request for a brand new key payment-12 all arrive within the same instant, before any of them has finished. Only one of those 10 should actually run the work and charge the card. The other 9 must wait for that one call to finish, then all get its result back. The charge still runs exactly once, not 10 times.
Where you meet this
- Stripe (API v1): once a request with a key starts running, Stripe saves the status code and body and returns them for retries with the same key and the same values. A retry with different values gets an error. Keys can be removed once they are at least 24 hours old. Stripe's newer API v2 works differently: it finishes the failed work on a retry instead of replaying the saved answer. (Stripe idempotent requests)
- Amazon EC2 lets you pass a
ClientTokento calls likeRunInstances. A retry with the same token and the same values does nothing new. Inside the same Region (or Availability Zone, for some calls), a retry with other changed values fails withIdempotentParameterMismatch. (EC2 idempotency) - The IETF had a draft for an
Idempotency-KeyHTTP header forPOSTandPATCH. The draft has expired and is no longer active, so it is not a published standard. Stripe uses a header with this name. (Idempotency-Key draft)
When to choose it: a request changes something (charges money, creates a resource, sends a message) and the client may retry after a timeout.
When not to: the request is already safe to repeat, like a read, or "set the name to Ana". Also decide what a second copy that arrives during the first one should get. Stripe does not save a result for a request that clashes with one still running, so the client retries later. This challenge makes the second caller wait and share the first result. Both are valid choices.
Common bugs
- Running the work again on a repeat request.
- Letting several callers for the same new key each start the work, instead of only one.
- Keeping a result after it has gone stale.
- Treating a result exactly at its time limit as still fresh.
- Removing the wrong result when the cache is full.
Keep it fast
A fresh saved result should need only a map lookup. Do not create a waiting object for a call that can return immediately.
Learn more
Stripe's documentation explains idempotent requests in a payments setting. Production systems often save results to disk too, so a restart does not forget them.