How do I tell whether a metric actually dropped or the data feed broke?
Check the shape of the drop before you check the business. Real changes are partial and continue over several days. Feed failures are total, start at a clean boundary like midnight or a sync window, and hit every metric sharing that source. If one source goes flat while an independent source covering the same events still moves, it is a pipeline problem, not a demand problem.
A break and a decline have different shapes
Demand does not fall to zero. When calls genuinely drop, they drop by some fraction, they drop unevenly across sources, and the drop persists with noise around it. A broken connector produces a cliff: full volume through 11:59pm, nothing after. The cleanliness of the edge is the first clue, because business activity almost never respects a clock boundary that precisely.
The second clue is correlation. If booked jobs, revenue and average ticket all go flat on the same timestamp, they are not three business events. They are one sync that stopped.
Three checks that take about five minutes
Before anyone speculates about the market, run these in order.
- Row counts, not metric values. Look at how many raw records arrived per source per day. A metric can move for a hundred reasons; a record count going to zero has one.
- Last-record timestamp per source. Every feed should expose the most recent record it delivered. If the newest call record is thirty hours old, stop investigating the business.
- An independent witness. Compare against a system that observes the same events through a different path: the phone bill against the call platform, the ad platform against your click log, the accounting system against the field service system.
Instrument the pipeline, not just the business
Most reporting stacks monitor outcomes and nothing else, which means the first person to notice a broken feed is an owner asking why Tuesday looked terrible. Freshness and volume monitors belong next to the business alerts: expected record count bands per source, expected arrival time, and an explicit alert when a sync returns nothing.
This is ordinary integration plumbing, and it is the cheapest reliability work available. A dashboard that cannot tell you when it was last fed is a dashboard you have to verify manually forever.
The failure nobody catches: partial success
Total outages get noticed. Partial ones do not. A sync that returns a success code but fewer records, a credential that expired for one location out of nine, a field that silently started arriving null after a vendor update. None of these produce a zero, so no threshold fires and the numbers just look a little soft.
The defense is per-source record counts displayed on the report itself, with a band for normal. A shortfall on one location while the others are normal is visible in one glance and invisible in a rollup. We build reporting that shows its own inputs for exactly this reason.
Topics: data quality · diagnostics · pipelines · alerting
Have a version of this question about your own business?
The useful answer usually depends on which systems you run and how they're connected. That's a conversation, not a blog post.