Nathaniel Ward

What you should remember when running A/​B tests

A/​B testing can help you make great leaps in optimizing your online marketing, but it’s not a panacea. It requires a lot of measurement, commitment and patience.

Peep Laja offers three useful warnings:

  1. Most A/​B tests won’t produce huge gains (and that’s okay)
  2. There’s a lot of waiting (until statistical confidence)
  3. Trickery doesn’t provide serious lifts, understanding the user does

This is spot on. Most of the tests I have run have failed to achieve any lift, while others were inconclusive statistically even after collecting considerable data. And his final point is critical: focus your testing not on the quick win but on how you can best convince your customers to buy.

If I had to add a fourth point, it’d be this: Run tests only to learn something, not simply for the sake of testing. What question are you trying to answer with your test, and how would the results lead you to do things differently in the future?


How to weed out bad job candidates →

When hiring, finding job candidates with the right technical skills is the easy part. Bryan Goldberg offers advice on how to separate the merely adequate applicants from the all-stars:

[I]f a candidate can’t even tell you why they liked their last job, or what they got out of their college experience, or any of the million other questions that speak to their basic humanness… Then no amount of experience will make them valuable.


Fire your SEO company and hire a writer →

The way to boost your site’s Google ranking, Paul Boag says, is to write quality content that your customers want to link to. Don’t waste time with consultants’ SEO voodoo.

“[I]t all comes down to content,” he writes. “If you create great content, people will link to it, and Google will improve your placement. It really is that simple.”


Non-profits need to fail more →

Seth Godin says non-profits need to be more innovative, which means they need to be more willing to accept failure:

Go fail. And then fail again. Non-profit failure is too rare, which means that non-profit innovation is too rare as well. Innovators understand that their job is to fail, repeatedly, until they don’t.

This is absolutely right. A non-profit group should do everything it can to better advance its mission. That may mean trying new and better ways to be even more effective. Fear of failure is the surest way to maintain the status quo.


Optimizing elevators →

Here is a typical problem: A passenger on the sixth floor wants to descend. The closest car is on the seventh floor, but it already has three riders and has made two stops. Is it the right choice to make that car stop again? That would be the best result for the sixth-floor passenger, but it would make the other people’s rides longer.

For Ms. Christy, these are mathematical problems with no one optimum solution. In the real world, there are so many parameters and combinations that everything changes as soon as the next rider presses a button. In a building with six elevators and 10 people trying to move between floors, there are over 60 million possible combinations—too many, she says, for the elevator’s computer to process in split seconds.