What actually happens when we hit a vendor's API rate limits?
Requests start failing with a retry signal, and a poorly built integration either drops that data permanently or keeps hammering until the vendor blocks it entirely. Designing around limits means queuing work, retrying with backoff, requesting only what changed, and treating the rate limit as one shared budget across every job, dashboard and user touching the same credential.
The limit is per credential, not per feature
This is the part that surprises people. Your nightly backfill, your live sync, your dashboard refresh and the third-party tool your marketing team connected last year may all authenticate with the same account. They share one budget. A one-time historical load can therefore starve the live connection that someone is actually waiting on.
The practical response is separation and priority. Where the platform allows separate credentials per workload, use them. Where it does not, put everything behind one queue that knows which work is urgent and which can wait until 3am.
Ask for what changed, not for everything
Most rate limit pain comes from pulling full datasets when a delta would do. Nearly every serious API supports a modified-since filter, a bulk or export endpoint, or an event feed. Using them turns a job that reads a hundred thousand records into one that reads a few hundred.
Cache the things that cannot change. Completed invoices, closed jobs and historical call records are immutable in practice. Re-reading them every night is pure waste, and it is the first thing to remove when a pipeline starts brushing its limits.
Backoff, jitter and the thundering herd
When a limit is hit, the correct behavior is to wait and retry with progressively longer delays. The detail people miss is jitter: adding a small random offset so that a hundred queued jobs do not all retry at the same instant and re-trigger the limit immediately.
Retries also need a ceiling and a dead-letter path. Work that fails repeatedly should land somewhere a human can see it, not loop silently forever. An integration with infinite retries and no visibility looks healthy right up until you notice a week of missing records.
Decide in advance what is allowed to go stale
Under sustained throttling something has to give. Decide which feeds degrade gracefully and which must stay current, and encode that as queue priority rather than discovering it during an incident.
Monitor queue depth and oldest-pending-item age as first-class metrics. Rising queue depth is the earliest honest signal that a pipeline is losing to its rate limit, and it appears days before anyone notices a stale number in a report. This kind of operational instrumentation is what separates a durable platform from a script, and it is why we treat throughput planning as part of integration design rather than an afterthought.
Topics: rate limits · APIs · reliability · architecture
Have a version of this question about your own business?
The useful answer usually depends on which systems you run and how they're connected. That's a conversation, not a blog post.