What stops an AI integration from creating the same record twice?
An idempotency key: a stable identifier derived from the source event and stored alongside whatever record it created. Before writing, the integration checks whether that key has already produced a write, so retries become harmless. Without one, any retry after a timeout creates a duplicate, because a timeout does not tell you the write failed. It only tells you that you stopped waiting for an answer.
A timeout is not a failure
This is the detail that surprises people. When an integration sends a create request and the connection times out, three things could have happened: the request never arrived, it arrived and failed, or it arrived, succeeded, and the response got lost on the way back. The caller cannot tell which.
The naive response is to retry. In case three, that retry creates a second record. Multiply by a busy Monday and you get the duplicate customers and duplicate jobs that make people distrust an integration they otherwise like.
Where the key comes from
A good idempotency key is derived from the event, not generated at send time, so a retry produces the same key.
- Source event identifier. The call ID, webhook delivery ID or message ID from the originating system. This is the best option when it exists.
- Composite natural key. Something like customer phone plus event timestamp plus action type. Workable, but fragile if any component can change.
- Content hash. A hash of the payload. Convenient, but it will treat two genuinely separate identical events, such as the same customer calling twice about the same thing, as one.
The duplicates idempotency will not catch
Idempotency protects against your own retries. It does not protect against two sources reporting the same real-world event, or against a dispatcher manually creating the record while the automation is mid-retry. Those need matching on the read side, usually on phone number and service address, and a rule for which record wins.
That is why deduplication is a two-layer problem: prevent your own repeats with keys, then reconcile the rest against the operational system of record.
How to tell whether your integration has it
Ask a single question of whoever built it: if we replayed yesterday's events through the pipeline right now, what would happen? If the answer involves cleanup, the writes are not idempotent. If the answer is "nothing, they would all be recognized as already processed," they are.
Replayability is worth having for its own sake. It is how you recover from a bad deploy, backfill a field you added later, and re-score a period after a rule change, without a database restore.
One more thing worth checking is where the key is stored. Keys held only in your own database are lost the day the pipeline is rebuilt or migrated. Writing the source event identifier onto the destination record itself makes the duplicate check survive a rebuild, which is the situation where duplicates are most likely to appear.
Topics: idempotency · duplicates · write-back · reliability
Have a version of this question about your own business?
The useful answer usually depends on which systems you run and how they're connected. That's a conversation, not a blog post.