Usability & A/B Testing Are Best Friends

Usability & A/B Testing Are Best Friends

Dennis van der Heijden
November 22, 2013

What are we talking about today: users or visitors, experiences or experiments, journey or funnels? Usability and A/B testing almost seem to be two different worlds. Usability is emotion, design and creating an experience while on the A/B testing side we test dozens shades of blue links to measure the increase of conversion or clicks, all statistics and analytics.

But pardon me sir, what about proving that design and usability improves the sign-ups, the amount of pageviews or the amount of game-hours? We can unite the two worlds of usability and statistics and actually improve the experience and the chance of a positive outcome significantly. Since 1 out of 3 tests get significant wins if you do a good preparation, while only 1 out of 7 A/B experiments are successful if you don’t.

You Can Not Unknow Your Conversion Rate

Dividing the number of sales by amount of unique visitors gives you a conversion rate and the total revenue divided by the unique visitors gives the revenue per visitor (RPV). These two metrics are the beginning of your worries. As soon as you know them and monitor them every month you will get the urge to improve them.

Trust me, I’m not a genius…

In 2008 when the world housing crisis hit, leads dried up for my business to business software company. Only using smart custom build software I was able to lift conversions by 106% in a couple of months. But only a year before that I figured out what conversion rate actually was and how it worked. The idea of making more money on existing traffic seems so obvious that you’re bound to miss it. It’s more logical to add Adword dollars right?

When nobody has traffic everyone goes to Adwords, bids rise and it becomes unaffordable. The only way to outbid your competition is to make more money on this traffic on your site. Double from 1% to a 2% conversion rate and you can double your Adwords bids to make the same revenue.

As soon as this really, really sinks in… there is no ‘unknowing’ the term conversion rate.

Conversion Rate Optimization Not Equals A/B Testing

You can get really obsessed with conversion rate optimization and learn that words, images, flow, color, timing, structure all play a part in it. But what it all comes down to in the end, is understanding the user. Understand how he thinks and behaves and try to stimulate a desired action and make that feel natural. Improving conversion rates is not the same as A/B testing… A/B testing comes after you understand the thoughts, actions and flows from users using tools like UsabilityHub offers. But even before that you need to understand that you need to solve a problem. This is the flow of increasing conversions on your or your client’s website:

  • Have a clear goal for the site
  • Measure the current goal (conversion rate, RPV, pageviews or engagement)
  • Set up a budget for improvements with a clear objective
  • Make someone responsible for this improvement
  • Create a chart to understand where in the process people drop out of the flow or funnel
  • Focus on one page and run a 5 second test & click-test, understand the problem of the page.
  • Write down a hypothesis of why this is happening
  • Come up with potential solutions
  • Run them again thought UsabilityHub’s tools
  • Finetune the solution
  • Set up an A/B experiment to validate this on your own audience
  • Take the winner of the experiment and make it permanent
  • Start again…

Website optimization teams are more effective when combining usability and A/B testers work closely together. Trying your new design with real users, using tools such as A/B testing software, before final implementation on your website is a very valuable part of the optimization process. Just don’t test without having any idea what the problem is you are trying to solve, start with usability studies and prove your solution with A/B testing statistics.

The fact that usability and A/B testing tools are now so readily available, easy to use and very affordable almost makes learning about them easier. Setting up a dummy headline test on any site without actually installing any code is possible in under 3 minutes. You should enrich you usability suite with the last part, running live experiments on your site has never been easier.

Originally published November 22, 2013 - Updated April 15, 2019
Dennis van der Heijden
Co-founder and CEO of, Dennis is a passionate community builder and out of the box thinker. He spends his time innovating to make Convert Experiences better. Learn about his journey as an entrepreneur and leader on the SaaS Club podcast.
Guest Post Form

We have brought thought leaders, influencers, visionaries and veterans to our tribe. Now it’s your turn. If you have something worthwhile to share with a large community of savvy testers, go ahead and pitch your post idea. We’re listening.



[hclightbox id='5' text='Anchor text']