Conversion work has the best economics of anything in marketing and the worst survival rate as a programme. Most testing operations die within six months, and the pattern is consistent: they test trivial changes, call winners on day four, and conclude that CRO is overhyped.
Both failures are avoidable. Trivial changes produce trivial results because a button colour was never the reason someone didn't buy — meaningful lift comes from testing offers, value propositions, page structure, and the friction the team assumed was necessary. And calling winners early inflates false positive rates well beyond the 5% the test is supposed to guarantee, which is why the sample size has to be calculated and committed to before launch.
The other thing teams miss is where the conversion event actually is. In SaaS it's inside the product, days after the marketing site did its job — optimizing signup while activation leaks just acquires more people who churn faster. In e-commerce it's frequently delivery certainty and total-cost transparency rather than anything on the product page.
What decides outcomes here
Research before hypotheses
Guessed tests win about one time in ten. Tests built on session recordings, exit surveys and user testing win closer to one in three.
Calculate the sample size and commit to it
Peeking daily and stopping at first significance is the single most common methodological failure in commercial testing.
Losses are the point
About 69% of properly designed tests don't win. Each one tells you something true that contradicted a confident internal assumption.
Keep an insight library
Every test, hypothesis, result and conclusion written down. Within two years it's a genuine competitive asset and it stops you paying to learn the same lesson twice.