How to compare two attribution models on the same orders

If you want to know which attribution model suits your store, the only way to find out is to read the same month under two models and see what moves. Most tools won't let you do that.

The reason is architectural. Credit is calculated at collection time: the model is applied as data arrives, and the result is stored. Change the model and you've only changed what happens to data arriving from now on.

What happens when the model is fixed at collection time

• Switching gives you a break in the series: old months on the old model, new months on the new one
• You can't compare the two on the same orders, so you can't tell what the switch did
• Any comparison you do run mixes the model change with everything else that changed that month
• Reverting creates a second break, so nobody reverts

The result is that the first model anyone picked becomes the model, and every discussion after that is a defence rather than a test.

The alternative: store the journey, compute at read time

You keep the journey as it happened: every touch with its source, campaign and timestamp, tied to the customer and to the order. Attribution is calculated when you open the report.

A model then becomes a question you ask of stored data rather than a filter applied before storage. You switch, you switch back, and you read last month under five models in one sitting. The cost is more storage and a heavier query.

That's what's live in Flowfy today: the journey is stored per order over a 90-day window against a resolved identity, and attribution is computed at read time.

What one month under three models shows

Three things show up that no single column gives you.

The role each channel plays: Snapchat and influencers open journeys, paid search closes them.

How much the model choice is actually worth to you. If your budget split wouldn't change across those columns, stop the debate and go do something else.

And which channels barely react to the model at all. Direct is identical in every column because those journeys have one touch.

The practical move is to take the spread between each channel's highest and lowest figure. Only the channels with a wide spread are worth discussing. The rest you can fund without another meeting.

"The team will just pick whichever model flatters them"

Fair objection, and there are two fixes. Name the model next to every figure you publish, so it reads "ROAS 3.2 on last paid touch" rather than "ROAS 3.2". And fix one model for recurring reporting, keeping the others for investigation when there's a specific question.

"Computing at read time makes reports slow"

True. It's more work per query than reading a stored number. The trade is deliberate, because it's what makes reading an old period under a new model possible in the first place.

"Does switching change my revenue?"

No. Switching redistributes credit between sources. Revenue comes from your store's orders rather than from the model, so the total doesn't move and neither does the order count.