What happens to an AI workflow when call or lead volume suddenly triples?
Without backpressure, everything degrades at once. Queues grow, the vendor rate-limits you, retries pile on top of the original load, and outputs arrive after the moment they mattered. A well-built system defers or sheds the lowest-value work first and protects the time-critical path. The design question is not whether it can handle the peak. It is what it gives up when it cannot.
Load does not degrade gracefully by default
The dangerous part of a spike is not the extra work. It is the feedback loop. Requests start timing out, the system retries them, the retries add load, more requests time out, and the queue grows faster than it drains. Storm season for a roofing company, a heat wave for HVAC, or a single successful campaign can all trigger it.
Naive retry logic makes this worse. Retries need exponential backoff with jitter and a hard attempt limit, or they become an attack on your own vendors.
The second amplifier is human. When outputs stop arriving, people resubmit forms, rerun reports and open tickets, all of which add load to a system already past its limit. A spike plan should include what you tell staff, not only what the software does on its own.
Decide the priority order before you need it
Under load, something has to wait. Deciding what, in advance and in writing, is the entire discipline.
- Tier one, seconds. Missed-call alerts, live routing, anything a person is waiting on. Never shed.
- Tier two, hours. Scoring, enrichment, same-day coaching material. Delay freely; it catches up overnight.
- Tier three, batch. Rollups, trend recomputation, backfills. Pause entirely during a spike and resume after.
Queue depth is the number to watch
Error rate tells you what already broke. Queue depth and queue age tell you what is about to. If the oldest unprocessed item in the tier-one queue is older than the tier's time budget, the system is failing whether or not anything has thrown an error yet.
That is the single most useful operational metric to put in front of whoever owns the system, and it belongs in the same place as your other operational reporting rather than buried in an engineering tool.
A spike is also a consumption spike
Model calls, transcription and API usage all scale with volume. A runaway loop or a retry storm consumes a great deal before anyone looks. Put ceilings on actions per hour and alert on volume anomalies, not only on failures. An automation quietly doing ten times its normal work is a problem even when every individual action succeeds.
This is one of the reasons operational AI is an engineering discipline rather than a configuration exercise.
None of this needs exotic infrastructure. It needs the priority order decided in advance and written somewhere the on-call person can read it at six in the morning without having to reason it out.
Topics: backpressure · scaling · queues · reliability
Have a version of this question about your own business?
The useful answer usually depends on which systems you run and how they're connected. That's a conversation, not a blog post.