Every A/B testing software application that is available right now and will ever be created slows down your site. Period! Tweet This
A/B testing softwareSince we sell A/B testing software this is not easy to admit, but it’s part of the optimization effort. Every system you use to optimize conversion will have a negative impact on the website performance. It might be 0.05 or even 0.2 seconds, the difference in these different systems is for one part related to the softwares’ infrastructure and the other part.. your own budget for a tool (even 200ms does not seem to even affect conversion rates)
There are software solutions that replace your current DNS solution where all the traffic goes through their DNS and there are solutions where additional tracking beacons and tags are used to alter the user’s experience by presenting a variation. Both solutions will have a negative impact on site’s performance since there are either new routes introduced or new tags installed. The site performance is really important but the main A/B testing tools available on the market all manage to keep their load under 0.2 seconds (most of the time) and that is a great improvement compared to where we were a few years ago.
There is no research that shows a direct correlation between conversion decrease between a site that loads two seconds and the same site that loads three seconds, see some research that is related to this subject. But over three seconds there are strong signals that there is a negative effect on conversion rates around 6.7% for each second delay over that (see also
Update: August 29, 2013: Added link to new blogpost: Seven Out Of Ten of All A/B Tests Fail