Why a conversion report can't tell you what the earlier work was worth
Attribution shows you the path to a sale. How far you can trust it depends on the method, the data and the time window, and none of those can see everything. Work that influences a buyer months before they search, click or purchase leaves its traces somewhere else, if it leaves any, so a conversion report can't be read as the full account of what a brand does. When the visible path gets mistaken for the whole cause, budgets drift towards whatever sits closest to the checkout, and the work upstream goes hungry.
Gordon, Zettelmeyer, Bhargava and Chapsky ran into a close cousin of this problem in a 2019 Marketing Science paper. They took 15 advertising experiments run on Facebook, covering 500 million user-experiment observations and 1.6 billion impressions, and compared what the randomised tests showed with what the usual observational methods estimated for the same campaigns. The observational methods often missed. In about half the studies with a purchase outcome, estimates were off by more than a factor of three, usually on the high side, though not always.
Two of the four authors worked at Facebook, which supplied the experiments, so weigh the findings with that in mind. The paper's value is the yardstick: randomisation gives you something solid to hold the cleverer estimates against. It says nothing directly about attribution models in general or about design craft, and it doesn't show that the missing value sits with the earliest creative touch. That would need its own test.
For a marketing budget it sharpens one question. Which numbers in your report estimate what a campaign added, and which only describe who saw what before they bought?
Picture a mid-sized furniture brand that spends six months on a new identity and a new photographic style. In the quarter after launch, last-click reporting hands most of the credit to branded search and email, which is no surprise, since that's where people finish. The identity work gets nothing. Yet some of those branded searches may exist because someone saw the new look on a friend's feed, recognised it in a shop window or remembered it from a catalogue, and typed the name instead of a generic query. The report can't tell you how many. It can't tell you none, either.
Before the measurable visit, a brand may be teaching a buyer which differences in the category matter. Design, language and repeated exposure can help someone recognise a product and know what to ask of it. Those are hypotheses until buyer research or an experiment tests them, but they deserve to be taken seriously. A report that logs only the steps it can observe will miss contact across devices, conversations and untracked media, and missing observations aren't a zero return. They aren't a guaranteed return for craft, either.
So put a specific question to the work. What should it change: recognition, consideration, price tolerance, the share of enquiries that are qualified? Then pick a way to test exactly that.
Picasso and Braque spent the years before the First World War painting guitars, tables and faces from several angles at once, on the theory that any single view leaves something out. A campaign has the same problem. You can ask what people remember, what they do in a randomised test, what the sales data shows over a couple of years and what buyers say they needed before they chose, and each answer has its own blind spot. The useful part comes when you lay them side by side and see where they agree and where they don't.
A randomised holdout can estimate what one defined campaign added to one defined outcome. Econometric modelling can look at longer patterns, provided its assumptions hold. Interviews explain why a choice made sense to the person making it. None is complete alone, so write the time horizon, sample, exclusions and assumptions next to every result. When two measures disagree, don't average them. Find out why.
There's an asymmetry here that rarely gets said aloud. A measured channel can take credit for a sale that would have happened anyway, and an early brand investment can influence a buyer without ever appearing in the path. Those are different errors. The Facebook experiments show the first in one setting; the second needs a separate inquiry.
Telling a finance team that craft can't be measured gets you nowhere. Make a narrower claim they can check: this piece of work should move a named buyer response, within a stated period, against a credible baseline.
Take one large creative investment. Set a brand measure before launch, choose a comparison audience and agree up front when you'll read the result. Track recognition and consideration alongside direct response, and where the scale allows, compare against a campaign holdout. If familiarity rises and choice doesn't, look at the claim, the product experience, the reach and the timing before deciding what caused what.
Interviews need to do more than ask whether people like the design. Put the work next to plausible alternatives from the category and ask what each brand seems to value, what the buyer expects the product to do and which cues told them so. Then compare their answers with what the team meant to say. If people notice the beauty and take away the wrong promise, the craft did a different job from the one the strategy assigned it.
A follow-up purchase study can check whether that understanding survives the moment of choice. Keep the claim modest. One good test might justify investing in one asset for one audience, and it won't establish a universal return on beauty. What it does is give creative and performance teams something specific to argue about: each side names what the work was supposed to change, and what evidence would make them change their mind.
The claim for craft has to say what a buyer might notice and how you'd check. Someone entering an unfamiliar category often can't judge quality from a spec sheet, so they lean on cues. Interviews can show which cues they use and whether the visual system helps them spot a difference that matters. Those cues also have to line up with the product experience, because that's where the promise gets tested.
Careful presentation can signal care, but anyone can copy a look. The design earns its keep commercially when buyers notice the distinction and the organisation can deliver it. So test whether the work shifts recognition or consideration among likely buyers, then follow whether those buyers behave differently once they're in the category.
Craft traditions are useful here as a discipline: show the decisions behind the work and let people judge what those decisions make possible. The commercial claim still needs evidence from the market.
If a piece of work looks distinctive and does nothing for the audience it was made for, you want to find out early. The case for creative ambition gets stronger when it comes with a test of what the work is expected to change.
Useful measures include unprompted recall among prospective buyers, consideration in a defined category, response to a price change and the words new customers use when they describe the brand. Give each one a baseline and a time period. A single number moving is something to investigate, not proof of cause.
A brand that measures only the last recorded interaction will keep getting better at the activity nearest the sale. It can also test the earlier choices that decide what the buyer wants, so long as craft is held to the same standard of proof as the rest of the budget.
