Top 6 Fundamental A/B Tests Necessary to Achieving Your Goals

Top 6 Fundamental A/B Tests Necessary to Achieving Your Goals

Eyal Katz
Written by Eyal Katz
July 21, 2015

When Conversion Managers think of goals, they naturally gravitate towards ROI focused conversion metrics. That’s why many A/B test experiments focus on producing lift from small changes to on-page elements like images, videos, or buttons, that are directly responsible for directing traffic to conversions. But changing these elements can have an effect on more than just CR. If conversion managers take a minute to step back and look up from their testing dashboards, they may find that taking a more holistic approach to A/B testing can prove to advance their goals.

Because deep down, deep deep down, CROs are philosophers just as much as they are scientists. As philosophers, CROs need to get out of their own heads and think about the meaning of their A/B tests and about their role within the greater scheme of things. What does it mean when I test a CTA button color? Why am I testing this element over another? What is the overall effect on the entire website? These questions are an essential part of creating your testing strategy from which you can further develop experiments that will be focused on your goals.

While you mull that over, I’ve gone ahead and shared with you my list of A/B tests that take a more holistic approach to optimization and are necessary to achieving a website’s goals. These are not tests that will show up in your dashboard. Rather these are the fundamental that lie behind your most effective tests.

[Tweet “Deep down, CROs are philosophers just as much as they are scientists.- @AdNgin1”]

6 AB 1

1. The Inbetweeners

We’ve all tested them. It’s a no-brainer. Forms, buttons, checkouts, etc. these are the elements that directly drive conversions, “the moneymakers”. The motivation for testing these elements seems obvious.

But one of the most important principles when taking a holistic approach is to re-examine and question the obvious. Examine what all your moneymakers have in common and how they differ from the non-moneymakers, and then define what makes them moneymakers. Draw up three columns on a whiteboard. Place all the elements considered to be moneymakers in one column. Place all the elements who don’t fall under the definition in another. What’s the third column for? All the elements that don’t perfectly fit in either column should be placed in the third column.

In addition, to your regular tests on the moneymakers, also test these inbetweeners. you may find that they produce surprising results.

2. Leaky Funnels

Leaky funnels need to be plugged to improve CVR. Pretty simple. But, as was the case in the section above, a holistic and philosophical approach to A/B testing your conversion funnel can prove surprising. To borrow from the world of real estate, if you’re unfortunate enough to find a leak in the walls of your home, finding its source and fixing it can be more complicated than it seems.

For an experienced plumber, the source of the leak may seem obvious at first while an inexperienced contractor may find it more challenging. Experienced or not, a plumber that takes her job seriously would take a step back and not be too proud to ask a few questions. Perhaps even ask to see the blueprints of the house. Otherwise, she may overlook a fundamental problem that will cause the leak to re-appear.

So, before you try to duck tape your conversion funnel leaks. Take a step back and remember that every website and every audience is unique and the causes and solutions for the leaks might not be what you expect.

[Tweet “Each website and audience is unique with unexpected causes and solutions for the leaks. – @AdNgin1”]

3. Micro Moments

A micro-moment encapsulates the various elements that are involved in creating an experience that is less constrained and more agile than before.

Not just a new buzzword, micro moments are for real. Mobile traffic is rapidly increasing and micro moments relate directly to the way mobile users engage your website. Bottom line, users don’t interact with your website in a linear way. Their interactions are more fluid in nature and less structured. Instead, interactions are fragmented into a series of “intent rich” micro-moments.

A micro-moment encapsulates the various elements that are involved in creating an experience that is less constrained and more agile than before. When A/B testing micro-moments, the researcher should examine holistic experiences. In other words, consider how all your moments combine and interact to create your user experience. Then test micro moments against one another. One thing to keep in mind when testing micro moments is that just as the user interactions are changing so are the conversions or rather micro-conversions.

6 AB 2

4. Relationships

This one’s particularly apt for the new guys on the block. Unless you’re a recognized brand, the trust your website induces with users can make you or break you. How do you test for Trust? There are various ways.

Like testing for returning traffic vs. new traffic or A/B testing the content and access of your about page, for instance. Thoroughly examine your funnel, as suggested in #2, you can find that you are missing trust signals, which can be a fundamental problem for your conversion funnel.

5. People are not People

It’s easy to focus on testing the on-page elements of the website and forget about your audience. With SEO becoming increasingly focused on bounce rates and user experiences, and every page functioning as a landing page, it’s increasingly important to test how various funnel personas engage your pages. Testing your funnel personas can also help adjust the persona profile you are using and improve the overall experience of the website as well as conversion rates.

6.  Buy Now or Please Come Again

With CROs focused on improving conversion rates and marketeers increasingly focusing on engagement, it’s only a matter of time until these two schools of thought collide. Most CROs would like to see users move down the funnel as quickly as possible but marketing would have you believe that time on page and pages per visitor are increasingly important metrics that will eventually favor the bottom line. This may go against every fiber in your being but test for longer conversion funnels that get users more involved with your content and may even increase your conversion rates. If it doesn’t work , you can always say I told you so.

The goal of a test is not to get a lift, but rather to get a learning. – Dr. Flint McLaughlin of MecLabs.

The thing about A/B testing is that it requires time and patience for it to be successful. But out there in the real world there are anxious clients and nervous bosses that want to see ROI from every A/B test as soon as possible. Using a holistic approach from the start can eventually reduce the time you spend performing tests and get you the lift you desire quicker. If that’s still not fast enough for them try using a quote from Dr. Flint McLaughlin of MecLabs: “the goal of a test is not to get a lift, but rather to get a learning.” Good Luck.

Obviously, there are more tests that could be added to this list. If you have tests that are your “go to” tests for achieving your goals, list them in the comments.

Originally published July 21, 2015 - Updated April 15, 2019
Eyal Katz
Eyal Katz is the Marketing Manager at AdNgin, an Adsense revenue optimization platform that uses a proprietary multi-arm bandit algorithm to create increased revenue for publishers.
Guest Post Form

We have brought thought leaders, influencers, visionaries and veterans to our tribe. Now it’s your turn. If you have something worthwhile to share with a large community of savvy testers, go ahead and pitch your post idea. We’re listening.



[hclightbox id='5' text='Anchor text']