Why Tests Fail and why that’s a Good Thing

Not too long ago, the Convert Academy hosted a webinar presented by Nazli Yuzak, a Senior Digital Consultant at Dell who was initially working in the optimization industry prior to joining the folks at Dell. She first worked at a global education company before venturing into the e-commerce setting, with  both stint swallowing five years of her life. You can still catch more details of her presentation here.

What is a Failed Test?

Nazli was perhaps better placed to speak about A/B testing software” href=”https://www.convert.com” target=”_blank”>A/B tests, and why they fail, given her enviable experience in the industry. The first session basically focused on the question: ‘What is a failed test?’ To start with, it would be great to understand what a failed test is in the first place. Let’s measure the time where you launch your test. You spend time analyzing your website, but looking at the numbers, the performance doesn’t mean anything much to you.

So, where could you have gone wrong?

You could be at the end of the test and you have found no conclusive results: it’s a flat number or nothing of significance, hence a failed test. However, things would be much better if you can maybe gain some insight from it.

Again, if you happen to have reached the end of your test and you’re not able to gain any insights whatsoever, which can help you come up with your next test idea or some kind of a next step to improving your website, this can also be deemed a failed test. Reason is because you have no insights to work off of, and there is nothing to show for the time invested in testing, not to mention another two or three weeks of planning.

A test could also be said to be a failed test if at the end of the test you do not have an action item that anyone can use or gain from that experience. For example, if you cannot have something concrete such as learning from the test that you should be making your CTA buttons larger, then this is also a failed test. Now that we know what a failed test is, how is this a good thing?

A Call to Action

Assume you’ve done a test and gained some insight from it. Take an example that still regards CTA buttons. After the end of the test, you’re probably able to establish that the main call to action button should always be the most prominent element on the page.

It may sound simple but there are times you might be able to get this out of the test, but it doesn’t get communicated to the rest of the organization clearly. You may have a 9% lift in conversion but you lack the insight to share with the rest of the organization. Again, with regard to the call to action buttons, every time you’re designing a page and you’re trying to direct people to an important call to action, you need to make sure that the CTA button is the largest element on the product page, as compared to other CTAs.

This is an action that needs to be institutionalized, an action that needs to be acknowledged, and an action that needs to be applied from that point onward in all projects that apply. This way, you’ll be ensuring that you do not repeat the same mistakes, and that you’re learning from it, and institutionalizing it in your learning.

You can check out the webinar in its entirety here.