Idempotency under 100 competing requests: a reproducible in-process experiment
An observable test for duplicate mutation suppression, tenant isolation, mismatch rejection, and retained errors—plus what a production persistence layer must add.
Suppose a client sends an HTTP mutation and times out. It may be tempting to generate a new operation ID and retry, but the first mutation may have committed. The right question is whether the server can safely recognize a retry as the same logical operation.
This behavior was reproduced using a small dependency-free Node.js test harness. The source and test fixtures are at ZombieMutex Idempotency Engine. This is synthetic, in-process code; it is not a database-backed distributed production service.
Setup
The implementation accepts a tenant ID, one operation key, exact request bytes, a retention period, and a callback representing the side effect. It computes a SHA-256 fingerprint of the request bytes. For the same tenant/key/fingerprint, every caller receives the first execution's Promise rather than invoking another callback.
The key detail is ordering: the registry inserts the pending Promise in its map before the asynchronous callback starts. Within one process, there is no second window for a fresh caller to claim the same key.
Run it
The main test schedules 100 calls for the same logical request:
The code is available verbatim in the public repository, including the test's actual request string.
Observed local outcomes
| Test case | Verified result |
|---|---|
| 100 concurrently scheduled identical calls | Side-effect callback executed once |
| Same key, different request bytes | Conflict rejected |
| Same operation key in two different tenants | Separate executions allowed |
| Failure with an uncertain commit | The failure is retained for later calls during the retention window |
| Key reused after expiry | A later execution becomes possible |
| Invalid key or TTL | Rejected before dispatch |
All six local tests passed in Node.js 24.21. These outcomes describe logic, not network throughput, tail latency, crash consistency or DB isolation guarantees.
Where the simple model fails
The local map is neither shared between service instances nor persistent across restarts. Two workers using different maps will both execute the callback. A process crash after an external payment is accepted but before a durable result is recorded may also leave an ambiguous outcome.
A production design needs a transactionally enforced claim for the tenant-scoped operation key, a stable request fingerprint, an explicit in-progress state, a durable terminal outcome or reconciliation status, a documented retention policy, and an outbox or downstream idempotency support for external effects.
The public repository contains an illustrative PostgreSQL table schema. It is intentionally not presented as a complete claim-and-commit transaction or a tested real database implementation.
The next useful experiment
Stand up PostgreSQL, send two requests in independent worker processes, and inject failure after the business mutation but before response persistence. Check whether the system can recover an unambiguous operation status without duplicating effects. That would provide evidence beyond the current in-memory reference.
You can also explore the shareable retry simulator and the companion HTTP timeout explanation.
Provenance: A new AI-assisted ZombieMutex reference repository, with measured pass/fail assertions from its local test suite. No customer transaction or production exactly-once guarantee is claimed.
Continue exploring
Related technical notes with overlapping topics.