Convert recently sat down with Dan Croxen-John, CEO of award winning AWA Digital to get his take on why conversion superstars, like Amazon.com, get the most out of their A/B testing. For the most part, Croxen-John observes following four common problems with websites:
- They’re finding it more difficult to get new traffic
- They’re drowning in Analytics data
- They’re bombarded by new, unproven marketing techniques
- They’ve got no time for split testing
But, Croxen-John has found that with his proven approach to A/B testing all of these issues can be resolved and they can see a 20% uplift in their annual profits. His five steps to getting the perfect A/B testing in line can help your site get new traffic, get data under control, eliminate the need for more marketing tools, and produce a test that’s time and cost effective. Here are the steps you can follow to get all of these benefits from your next A/B test.
Step 1: Getting Set-up
The foundation to a successful A/B testing is getting the right tools in order, and the most important tool you’ll put into use is your mindset. Of course confidence and self-assurance help with any task, but Croxen-John says that it’s important to have an “I don’t know mindset”. Relying too much on conjecture or general opinions about what makes a website great is going to take you away from what’s most important: your visitors.
A/B testing is about optimizing your visitor’s experience and zeroing in on their intent for coming to your site. Starting with the assumption that you don’t know what’s best for them, but being willing to find out, will help you to keep your research centered on the customer. That’s a big part of the reason Croxen-John stresses using online surveys and recruiting testers from your site visitors – it helps to keep everything focused on knowing your customer better.
Step 2: Use Google Analytics to find out how, not why
Some readers might be wondering why Dan Croxen-John doesn’t advocate using Google Analytics for research, but for him, relying on Google Analytics for understanding your visitors is more “hopeful than helpful.”. He says,
“As Google Analytics is becoming bigger and bigger in functionality and what it can report, it becomes increasingly difficult to find out what the critical areas are.”
What Google Analytics can help sites determine is the areas where A/B testing needs to be applied first. He believes that GA data should be used to place each of your site’s pages somewhere on this chart:
When you start to compare your “product/service page” views against the “add-to-basket/form completion” data, you can easily zero in on pages that are successful, pages that are failing completely, and pages that are holding back great products that sell without lots of page views. This is where you can use A/B testing to start playing around with what might work better.
Step 3: Recruit Qualified Users
The best way to understand your customer’s motivations when you are researching potential A/B testing is by pulling from your visitors. One way to do this is through an email blast to current customers, but this won’t catch a visitor right when they are about the begin their journey with your site. Offer them an incentive to answer some questions that will determine if they are a target customer and then ask if they will participate in Remote Moderated Usability Testing.
By using a Remote Moderated Test, you can have a real-life user vocalize their thoughts as they move through the site. This allows you to ask specific questions addressing pages falling into your “underperforming product” category, and what Dan Croxen-John thinks might be most important as the final question: “Is there anything today that nearly stopped you from ordering?”
Step 4: Data and Optimize Your Plan
The impulse to test everything, or to allow additional suggestions to come up after you’ve started combing through data and using Remote Moderated Testing can be hard to ignore. But, for Croxen-John getting at the best test is all about separating out your strong evidence from your weak evidence–what is the data really saying? Once you get at that, you can create a strict plan for what needs to be tested and what can be focused on at a later date.
“This is where taking a moment to think about how you can improve the split test design using a various forms of feedback really get the absolute best result.” – Dan Croxen-John
Step 5: Creative Execution
Once you have your research and the specific areas you are going to test, creative execution comes into play. Dan Croxen-John says, “This is where taking a moment to think about how you can improve the split test design using various forms of feedback really get the absolute best result.” For him, that means a somewhat labor intensive process of creating the page you want to test.
The chart shows the process, starting with a wireframing of how the page you want to test could potentially look, you will then get feedback on this in progress version, moving onto to artwork and feedback stage, and once you feel that the page would be ready to go up on the site, then put it through a test.
Some users who love using A/B testing might feel that this is too much, but Dan Croxen-John’s methods are all about “bold testing, not tweaking.” For him, the test should be clearly focused on your customers, a specific problem, and offer up a viable solution before you even get to the A/B testing in order to get the most out of it. For more information about the webinar, you can watch its entirety here.
Or you can download the presentation’s webinar: