Trends over precision

“The human mind seems to focus more easily on stocks than on flows. On top of that, when we do focus on flows, we tend to focus on inflows more easily than on outflows. Therefore, we sometimes miss seeing that we can fill a bathtub not only by increasing the inflow rate, but also by decreasing the outflow rate. Everyone understands that you can prolong the life of an oil-based economy by discovering new oil deposits. It seems to be harder to understand that the same result can be achieved by burning less oil.”

• Donella Meadows, Thinking in Systems

If you’re looking at a number, and it’s at a single point in time, it doesn’t indicate how well you’re doing in any meaningful way.

This rule extends to comparing 2, or 3, or 5 things.

Getting stuck on one thing working well, or one thing working better than another is a common trap.

Consider these examples:

Post A got more likes than Post B.

That’s all that happened. It doesn’t mean anything else in isolation.

The click-through rate on video was better than the posts with just graphics.

This isn’t useful either.

I got more sales from the email sent by this partner, than this partner.

Getting warmer. It’s useful, but in isolation not as useful as you would think.

The part that’s missing is time.

If something is consistently happening - video outperforming pictures, more sales from one kind of partner than another - that’s when we start paying attention and noticing what’s working.

A/B testing is running two similar offers with one key difference. It might be testing one headline over another, or a long offer page versus a short one. In startup circles - it’s often seen as a shortcut for learning what’s working from a marketing perspective.

The challenge with this is sample size. If you run Offer A and Offer B to 1,000 different people, and B gets 80 orders and A gets 55 - it still doesn’t tell you much in isolation. It’s also expensive and time consuming to get 1,000 people to see your offer. If you’re starting out - it’s hard just to get to that point, let alone repeat it over a number of weeks and months to understand if B is actually better than A and why.

Well funded scale-ups - with hundreds of thousands of advertising dollars to test offers against new people each week can test this - the rest of us can’t.

The other problem with obsessing over exact numbers and comparing them to other numbers is that it distances you from understanding your customer. You want to get as close as you can to understand what they’re saying, thinking, and feeling - focusing on 76 clicks versus 34 clicks is the wrong thing. It’s not even the same ball park.

I can say this because I learned the hard way. I spent years building complex pivot tables that highlighted just how well our agency was spending our client’s media budgets. The reality is it’s a very small piece of a much larger puzzle to solve.

Tom is a friend I kept in touch with after meeting him in such an agency. He’d sold the agency to a bigger one, and was onto his next thing. It’s a brick and mortar golf centre. People come and smack golf balls, and they also offer an online coaching program for amateurs to improve their abilities and track their progress. It’s going really well and he’s the happiest I’ve seen him.

I asked him what he loves about it the most:

“I used to love being able to run marketing from a keyboard and deliver conversions and optimisations that put good money in the bank. But when you see someone discover our centre, come in for the first time, and begin enjoying themselves - that kind of feedback is unique and something that you don’t get with digital.”

Remember - you want to look them in the eye.