How much history should we load when we connect a system for the first time?
Enough to cover one full cycle of whatever you are measuring, plus one comparable prior period. For most service businesses that means roughly two years - a year of seasonality and a year to compare it against. Load less and every trend is indistinguishable from noise. Load everything and you spend the first weeks of the project fighting data that predates your current field structure.
Two years is a rule of thumb with three exceptions
The two-year default exists because seasonal service businesses need a full annual cycle to say anything about a trend, and a second cycle to say whether this year differs from last. A single winter tells you nothing about whether demand is growing.
- Long sales or job cycles. If a quote routinely converts months later, extend the window until it comfortably contains the full lag.
- Recent structural change. A new operating system, a rebrand, an acquisition or a price book overhaul can make older data actively misleading.
- Customer lifetime questions. Retention, repeat rate and lifetime value need more history than campaign analysis does - often four or five years.
Old data is not the same data
The failure mode nobody anticipates: your job types, lead sources, service categories and status codes have changed over time, usually without a record of when. A chart that spans a taxonomy change looks continuous and is not. A category that appears to collapse in March may simply have been renamed.
Before loading history, ask the longest-tenured person in operations when major field or process changes happened, and store those dates as annotations. A boundary marker on a chart is worth more than three extra years of history, because it prevents confident conclusions drawn from an artifact. This is the unglamorous reality behind any revenue trend analysis.
Backfill is a different job from live sync
Treat them as separate systems that happen to share code. A backfill is one large, throttled, resumable run with a checkpoint so a failure at eighty percent does not restart at zero. A live sync is small, frequent and latency-sensitive. Sharing a rate limit budget between them means the backfill will starve the sync, so schedule the historical load for hours when nothing else is running.
Store the raw responses from the backfill somewhere cheap and durable. When you later discover a mapping error, replaying stored payloads takes an hour. Re-pulling two years from a vendor API takes days, if the older records are still available at all.
What to do with the era before your current process
Do not force pre-change data into your current model. Load it, mark it, and treat it as a read-only archive that answers questions of the form "roughly how big was this then" rather than feeding operational reports.
The alternative - retroactively remapping old categories into new ones - creates a dataset that looks authoritative and quietly encodes a series of guesses. If you do remap, keep the original value alongside the mapped one so anyone can check the work. Every serious business intelligence setup preserves the raw value next to the derived one.
Topics: backfill · historical data · seasonality · implementation
Have a version of this question about your own business?
The useful answer usually depends on which systems you run and how they're connected. That's a conversation, not a blog post.