Skip to main content

Our integration was down for a day. How do we recover the events we missed?

APIs & Data Published November 20, 2026
Short Answer

Do not count on the vendor's retry queue — most give up after a handful of attempts inside a short window, so a day of downtime means the events are gone. Recover by reconciling instead of replaying: query the source API for every record modified during the outage window plus a safety margin, upsert those, then compare record counts on both sides for that window before you declare the gap closed.

Retry windows are shorter than outages

Vendor retry policies vary widely, and most are designed for a brief blip rather than a real outage. Several disable an endpoint automatically after a run of consecutive failures, which means that even when service returns, nothing is being delivered until someone re-enables it in a settings screen nobody remembers exists. Checking that the endpoint is still enabled is step one of recovery.

Reconcile, do not replay

Replaying a delivery log only works if the vendor keeps one and exposes it. Reconciliation works everywhere the API supports filtering by modified date, which is nearly everywhere. You ask the source what changed between two timestamps and you apply all of it.

Widen the window on both ends. Start the sweep before the outage began and end it after recovery, because clock skew between systems and records written late by background processes both push events outside the naive boundary.

Rate-limit the sweep deliberately. A recovery job that tries to catch up as fast as possible will throttle the live pipeline that just came back, which turns a one-day gap into a two-day one. Recovery is not an emergency in the technical sense; it is a backfill with a deadline.

Entities that cannot be swept

Not every object supports a modified-since filter. Child records — line items, notes, appointment history — often do not. For those, sweep the parent objects that do support it and re-pull their children. It is more requests, but it is the only way to be complete.

Where even that fails, you fall back to a full reload of a bounded slice. Plan for this in the integration design rather than improvising it at two in the morning.

Prove the gap is closed

Building the sweep before you need it is the difference between an hour of recovery and a week of arguments about whose numbers are right. It belongs in the same monitoring layer as daily reporting.

  • Count both sides. Records created in the window, upstream versus downstream. Equal counts are not proof of correctness but unequal counts are proof of a problem.
  • Spot-check the boundaries. The first and last records in the window are where off-by-one errors live.
  • Re-run the sweep once. A second pass should produce zero new rows. If it produces rows, your watermark logic is wrong, not just your recovery.

Topics: webhooks · outage recovery · reconciliation · sync design

Have a version of this question about your own business?

The useful answer usually depends on which systems you run and how they're connected. That's a conversation, not a blog post.

Related Answers

People who read this also asked

Browse the Answer Hub →

AI is easy to access. Making it useful is hard.

Bluefrog makes AI useful by integrating it with the way your business actually works — your software, your calls, your customers, your marketing and your revenue.

Technology development since 1997 · AI integration platforms since 2001