Time your A/B tests to get best results

Time your A/B tests to get best results


Every time you run a test, think about things you can do to reduce noise- or uncontrollable effects on your data. One thing that is often overlooked is fluctuations caused by the day of the week. Think about it. Many sites enjoy significantly more traffic and more sales on weekdays, while others perform best on the weekends.

<a title=Web site visitors tend to be very different on the weekends and evening, than mid day during the work week. I have seen a lot of activity and responses come from senior management of very large companies. Rarely during the day. Weekends are when these time-stressed executives get to surf.

Say you run a test starting on Monday. By Thursday, you could have a substantial amount of data, pointing to a clear “winner.” The inexperienced tester would begin celebrating prematurely. Along comes the weekend, and what you thought was the winner could end up falling behind. How to avoid this? Always run your tests by the calendar week (yes you can schedule tests in Convert’s A/B testing software). This essentially removes the weekday/weekend fluctuations from affecting your end results.

Originally published August 12, 2014 - Updated April 15, 2019
Dennis van der Heijden
Co-founder and CEO of Convert.com, Dennis is a passionate community builder and out of the box thinker. He spends his time innovating to make Convert Experiences better. Learn about his journey as an entrepreneur and leader on the SaaS Club podcast.
Guest Post Form

We have brought thought leaders, influencers, visionaries and veterans to our tribe. Now it’s your turn. If you have something worthwhile to share with a large community of savvy testers, go ahead and pitch your post idea. We’re listening.



[hclightbox id='5' text='Anchor text']