Peep Laja identifies eleven common A/B testing mistakes (his headline says twelve, but the article is missing a ninth item):
- A/B tests are called early
- Tests are not run for full weeks
- A/B split testing is done even when they don’t even have traffic (or conversions)
- Tests are not based on a hypothesis
- Test data is not sent to Google Analytics
- Precious time and traffic are wasted on stupid tests
- They give up after the first test fails
- They don’t understand false positives
- They’re running multiple tests at the same time with overlapping traffic
- They’re ignoring small gains
- They’re not running tests at all times
In my experience, the first, fourth, and tenth mistakes are easiest to make. I’ve made them myself in my impatience to get a result, my desire to just “try something,” or my desire for a big lift.
But cutting corners to get a big lift as quickly as possible doesn’t teach you anything you can use in the future—and learning is the most valuable takeaway from any test.
Read the whole thing.
What mistakes have you made when testing?
Upworthy succeeds in driving piles of traffic not because because it uses some sophisticated new technology.
Upworthy succeeds because it recognizes that its readers are human beings, not abstract “traffic” or “eyeballs.” This is an old technique applied to the web.
As Derek Thompson explains in his report on Upworthy’s success, its writers select stories that have emotional appeal and craft headlines to pique our curiosity. They combine this with optimization techniques to find out what works best:
Upworthy has mastered the dark viral arts with a unique blend of A/B technology and lily-white earnestness. The staff scours the Web for “stuff that matters,” writes multiple headlines for a test audience, selects the top-performer, and blasts it out on social media. It’s a deceptively simple plan that’s devouring the Internet, one Facebook Newsfeed at a time. The site nearly surpassed 50 million unique visitors in October, which suggests traffic comparable to giants like Time.com, and Fox News. …
What’s the “secret”? An entertaining slideshow of Upworthy’s headline-writing strategies last year repeatedly references the “curiosity gap.” The idea is both to share just enough that readers know what they’re clicking and to withhold just enough to compel the click.
That linked slideshow is definitely worth reading through.
Adam Penenberg challenged the white-hat hackers at SpiderLabs to penetrate his digital life. They found fewer ways to break in than at their corporate clients—but once in, they had access to everything:
With me, however, there were fewer paths that could lead to the mother lode: my laptop, email, bank, social media accounts, and home. Once in, though, his team found few firewalls protecting my data, and mostly in the form of a pastiche of passwords and log-in credentials. These, I quickly learned, were not secure.
Are you looking to raise money online? These four tips may be helpful:
- Make sure people can donate on your web site
- Build an effective e-mail program
- Make sure you’re mobile-ready
- Use social media to build relationships, not to drive gifts
But the number one trick to being a great online fundraiser? Assume you have no idea what actually works.
I elaborated on these points in a presentation last week at the Leadership Institute. My full slide deck is below.
The adage “learn from your mistakes” is a bit misleading, Mig Reyes argues. Instead of trying to fail, we should just try:
The makers of our world would be better off mimicking scientists with their work. Harp on deliberate practice. Reinvent their processes daily. Share every discovery. And most importantly, try new things often…
When everything’s an experiment, you shed the fear that comes with trying new things.
Consistent experimentation is the key to success, whether in marketing or any other field.
The three most important lessons online fundraisers should keep in mind are:
- Don’t trust your gut instincts
- Test everything
- Always ask, “Can I do better?”
That was the message I presented last night, using examples from my work at The Heritage Foundation, at an event sponsored by Empower Action.
My full slide deck is below.
It’s easy for you to fall into bad habits as an online marketer, especially when it comes to data analysis. And your decision-making using misunderstood data may be just as bad as your decision-making based on gut instinct.
The tools you’re given of of little help. They bury the useful data deep in report pages.
My e-mail tool, for example, will tell me how many times a fundraising message was opened or clicked. It requires more work to determine how many donations that message generated and for how much money—and those are the numbers that matter in a fundraising appeal.
Three terrible data practices make Loren McDonald’s list of a bad online marketing habits: (more…)
Kevin Williamson explains Starbucks’ business model in The End Is Near And It’s Going to Be Awesome (which you should definitely buy):
New York City does not have much of a publicly financed public restroom infrastructure as such. What it has is Starbucks, a privately financed public restroom infrastructure with a very successful for-profit coffee chain attached to it. Maybe you don’t think of Starbucks that way, and Starbucks certainly doesn’t think of Starbucks that way, but residents of New York City apparently think of Starbucks that way.
Good typography can make a great text even greater, but it’s often hard to know where to begin.
Fortunately, we have Matthew Butterick’s Practical Typography, which includes a summary of 26 useful typography tips. His entire book, which includes discussion of everything from type faces to page layouts, is a great resource and worth a read.
At first blush, Moneyball has nothing at all to do with marketing. After all, it’s a book about baseball. But there are three important lessons online marketers can draw from the book.
1. Start with a clear objective, and focus all your activity on that
To ensure the Oakland As won more games than anyone else on a shoestring budget, general manager Billy Beane identified those metrics, like on-base percentage, most correlated with scoring runs and winning games. He then ensured his lineup included players that could get on base, and discouraged tactics like bunting and stealing that kept players from getting on base.
Online marketers should use a similar goal-oriented approach. Start with your highest measurable objective (sales or brand awareness, for example), and then identify those activities (sending e-mails or running a particular kind of ad) and customer behaviors (like commenting or browsing a product page) most correlated with achieving that objective.
And since you no doubt have limited resources, you should focus on these activities that give you the most bang for your buck—and stop doing those things with low or even negative correlation with success. (more…)