The failure window cannot be wished away
A caller sends a request. The server commits it. The response disappears. From the caller's point of view, nothing is known; from the server's point of view, a message now exists. Retrying with a fresh identity creates a duplicate precisely when the first attempt succeeded.
Idempotency turns the caller's intended business operation into a shared recovery handle. The request can be repeated until the caller learns the authoritative result.
Choose the key before the HTTP attempt
Good keys come from the event being communicated: an order-status version, invoice reminder sequence, appointment change, or authentication challenge. They must be stable across worker restarts and retry code paths.
A random UUID can be valid if it is created once, stored with the business event, and reused. A UUID generated inside the retry function is not idempotency; it is duplicate generation with nicer syntax.
Bind the key to material request content
The server should store a canonical hash of fields that define customer-visible behavior: app, channel, recipient, template, variables, and client reference. Object key order should not change the hash. Secret or transient transport metadata should not be part of it.
If the same key arrives with different material, return a conflict instead of choosing one silently. The conflict is evidence of a caller state bug, concurrent mutation, or unsafe key reuse.
Use an outbox on the caller side too
When a business transaction creates a message, write an outbox record in the same database transaction. A worker reads that record, uses its stored idempotency key, calls the messaging API, and records the returned message and request IDs. This closes the gap between committing the business event and attempting delivery.
Persist enough to investigate
- business event and version;
- idempotency key and canonical material;
- first attempt and latest attempt times;
- MessageHop request and message IDs;
- latest known lifecycle state;
- final business decision, such as cancel or resend.
Treat conflicts as reconciliation work
A 409 idempotency_conflict is not transient. Compare the stored operation to the new request. If the first operation is correct, preserve it and repair the caller. If a genuinely new customer-visible message is intended, create a new business event and a new key deliberately.
Idempotency continues downstream
The acceptance key prevents duplicate accepted operations. Queue tasks, provider attempts, callbacks, ledger events, and support replay also need stable identities. Reusing the original message and financial identities is safer than manufacturing a new send during operational recovery.