How do I set alert thresholds that don't turn into noise?
Derive the threshold from the metric's own normal variation instead of a round number. Look at several weeks of history, find the typical swing, and alert only outside it — then require the condition to persist or the underlying volume to be large enough to matter. Track how often alerts get acted on. An alert people ignore is worse than none, because it trains them to ignore the next one.
Round numbers are the problem
"Alert me if booking rate drops ten percent" sounds precise and is arbitrary. If your booking rate normally swings more than that day to day, the rule fires constantly. If it normally swings much less, the rule never fires until the situation is already severe.
The rule of thumb: measure the metric's typical day-to-day movement over a stable recent period first, then place the threshold outside it. The threshold is an output of the data, not an input from a meeting.
Three ingredients, all required
Most bad alerting systems have magnitude only. Adding the other two typically cuts alert volume dramatically without losing the events that mattered.
- Magnitude. How far outside normal the value sits.
- Persistence. How long it stays there. A one-day dip in a noisy metric is usually nothing; three consecutive days is a signal.
- Minimum sample. Suppress ratio alerts below a floor of underlying events. A close rate computed on four estimates should never page anyone.
Route to a person or don't send it
An alert delivered to a shared inbox is a notification. An alert delivered to a named person with a stated expected action is an assignment. Only the second one changes anything.
Route by what the recipient controls: unbooked bookable calls go to whoever coaches the phone, spend pacing goes to whoever can change bids, uninvoiced jobs go to the office. Call-level analysis is what makes the first route possible.
Measure the alerts themselves
Keep a simple log: alert fired, was it acted on, was it real. After a month you can see which rules are earning their interruption and which are producing reflexive dismissal. Rules below a reasonable hit rate should be retuned or retired.
Threshold tuning is ongoing work, not a setup task — seasonality, staffing and channel mix all shift the baseline. Treat it like any other operational system that needs maintenance.
Suppression rules you will want immediately
Threshold math is only half of it. The other half is controlling how alerts behave once they fire.
- Deduplicate. One alert per condition per day, not one per evaluation cycle.
- Respect hours. Nothing outside working hours unless a human can genuinely act on it then.
- Escalate on repeat, not on severity. A condition unresolved for three days deserves a different recipient, not a louder message.
- Group related conditions. Five campaigns pacing over budget is one message with five lines, not five messages.
Topics: alerts · thresholds · exception reporting · monitoring
Have a version of this question about your own business?
The useful answer usually depends on which systems you run and how they're connected. That's a conversation, not a blog post.