Every A/B testing software application that is available right now and will ever be created slows down your site. Period! Tweet This
Since we sell A/B testing software this is not easy to admit, but it’s part of the optimization effort. Every system you use to optimize conversion will have a negative impact on the website performance. It might be 0.05 or even 0.2 seconds, the difference in these different systems is for one part related to the softwares’ infrastructure and the other part.. your own budget for a tool (even 200ms does not seem to even affect conversion rates)
There are software solutions that replace your current DNS solution where all the traffic goes through their DNS and there are solutions where additional tracking beacons and tags are used to alter the user’s experience by presenting a variation. Both solutions will have a negative impact on site’s performance since there are either new routes introduced or new tags installed. The site performance is really important but the main A/B testing tools available on the market all manage to keep their load under 0.2 seconds (most of the time) and that is a great improvement compared to where we were a few years ago.
Update: August 29, 2013: Added link to new blogpost: Seven Out Of Ten of All A/B Tests Fail