Why a fixed alert threshold produces false alarms

The alert fired at 3am. Nothing was broken; the store doesn't sell at that hour anyway. It fired at the same time yesterday, and the day before. After the fifth one, someone on the team muted the channel. Two weeks later tracking actually broke, and nobody heard about it.

That story starts in the same place every time: a threshold with a fixed number in it, because a fixed number is the easiest thing to build and the easiest thing to explain.

Why one number can't serve two stores

A store doing 40 orders a day and a store doing 4,000 have nothing in common. Neither does the same store in Ramadan and in the second week of February.

So a fixed threshold fails in both directions at once:

• Too low for the large store. A destination can go entirely silent while total volume stays above the line, so a real break never fires.
• Too high for the small store. Normal daily variance crosses the line regularly, so the alert fires on nothing.

And the same store moves between those two states across a year. A threshold set correctly in February is set wrongly by Ramadan without anyone touching it.

Normal is a shape, not a number

What matters isn't volume. It's how far today sits from this store's own recent pattern, at that hour, on that day of the week.

A store that never sells between 2am and 6am shouldn't be alerted every night at 3. A store whose weekend is Friday shouldn't be alerted every Friday morning. A store in Ramadan shouldn't be alerted through the daytime lull, which we covered in the daily rhythm.

To do that, a detector needs three things a fixed threshold doesn't have:

1. A baseline from the store's own recent data, per hour and per weekday
2. A definition of your longest normal quiet period
3. A way to compare one destination against the others, so a single-destination failure shows up even when totals look fine

The check that needs no threshold

The most reliable single detector needs no tuning at all: compare your store order count against conversions delivered per destination.

Those two numbers should track each other. When they diverge, the size of the divergence is the size of the problem, whether your store does 40 orders a day or 4,000. It scales with your volume by construction, which is why it's worth running by hand if you have no alerting.

In Flowfy, the delivered log with per-event platform responses, revenue by source and conversions delivered per destination are live today, and that's what this check reads. Automated detectors with baseline-relative thresholds haven't shipped, and they're on the roadmap.

Don't send an alert without a riyal figure

An alert with no financial consequence doesn't get opened. Multiply the missing orders by average order value and put the result on the notification.

Two things follow. People open the notification, because a large riyal figure isn't something anyone scrolls past. And the alerts sort themselves, because the one carrying the largest number is the one you open first.

If the riyal impact sits below a level you'd act on, don't send it. An alerting system is judged on the break that got through without a notification, not on how many it sent. Every unhelpful one moves your team a step closer to muting the channel the important one will arrive on. Pick a channel that actually gets read, and in this market that usually isn't email: see a weekly digest.

The practical step