Today we announce another speed improvement for our clients. We already signalled that we moved…
The Technology Behind Convert’s A/B Testing Platform
Written by Clinton Skakun
July 14, 2015
As a web technology buff I love learning how web-based apps and services pull off, what appears to be, magic. Today I’m digging into the inner-workings of Convert.com to learn how their A/B testing platform works. I want to address some of the questions and potential concerns of web designers and developers who integrate Convert Experiments into their company’s website.
So how does Convert’s technology affect your site in terms of loading times and other 3rd party plug-ins you’re team has installed? Let’s start with the basics…
The basic end of the editor allows you to change text, update links, change background color and change positions of items on the page.
Setting up a Split Test from the Editor
Setting up your A/B test starts in the Convert Visual Editor. Once you see your website in edit mode, you quickly realize how easy it is to set up a test. Within minutes you can have an A/B or multivariate test ready to go. The visual editor allows you to change text, update links, change the background color and change positions of items on the page.
If you’re comfortable with CSS and Javascript, you can also add custom code to take your variants to the next level. There’s a lot more to the editor than I can cover in one article. You can learn more by going to the Convert Support Center and clicking around. A majority of the editors use jQuery to make edits. This is used in the same way on your website once your split test is published.
The Script Tag
It all starts with the <script> tag. And it’s what makes Convert’s set-up a no-brainer, even for non-developers. You don’t need to mess around with any server-side scripting or install packages on your host. If you’re a front-end designer, this is as easy as other services such as Google Analytics, KISSmetrics, etc. The script you get from Convert should go at the bottom of <head> tag. Lots of web designers like to put their scripts at the bottom of the page, before the end of the </body> tag. Either of these places will work fine for any other script, but not for A/B testing software. The script needs to load faster than any other element in order to make changes invisible to the visitor.
Credible sources tell us that keeping the web site loading times under 2 seconds is ideal for minimizing bounce rates. I prefer keeping it even lower, say 1.5 seconds or less.
Loading Time Concerns
Any web developer who reads this would instantly be concerned about time taken to load the scripts, the variation and HTML DOM updates before making it visible to the visitor. The reality is that this happens really blazingly fast, with the help of AWS, Cloudfront and other CDNs designed for serving static files. You’re sacrificing around <= 50 ms for the purpose of A/B testing, which is faster than CSS files load on a typical website.
[Tweet “Keeping the web site loading times under 2 seconds is ideal for minimizing bounce rates”]
Neil Patel tells us that keeping the web site loading times under 2 seconds is ideal for minimizing bounce rates. I prefer keeping it even lower, say 1.5 seconds or less. As long as your website is well optimized for speed and follows all of the best practices, you generally have little to worry about. Convert’s script alone will not cause your site any drastic loading time differences.
How Variations Are Selected
Variation rules and goals are loaded with the script file embedded in your page. Variations are loaded synchronously in the backend and tracking/goals are asynchronous loaded 50 ms; in the end, loading everything really fast. The first time a user visits one of your pages containing an A/B test, a random variation is selected and later stored in a cookie to remember the variation for the next time the user visits the page. Data is then sent back to Convert’s servers to report your tests.
Preventing a Blink
In order to instantly show the variation instead of the original content, the body is hidden until the variation loads (which happens in around 50 milliseconds). Once the DOM has been updated to the correct variation, the body element is made visible. The script is usually loaded in before the DOM loads but in some rare cases the original content may show instead of the variation. For this reason we hide the body until the variation is set.
Potential Effects on Other 3rd. Party Services
Convert’s embedded script works like any other service you embed in your website. Most, if not all, 3rd. party scripts have their own Javascript scope and enclose their library within this scope to prevent accidental globals. Usually, there’s one global variable exposed from the script, like _ga or mixpanel. This allows developers to access the service’s API methods.
For Convert’s script, the global convert variable is exposed. So as long as there are no other scripts using the global convert variables for other purposes, the script shouldn’t interfere with existing Javascript code.
Visitor targeting works by extracting information from the HTTP headers and info about the browser that can be accessed in Javascript.
Visitor Targeting
Visitor targeting works by extracting information from the HTTP headers and info about the browser that can be accessed in Javascript. This is basic data sent to the server when a visitor lands on your website. It contains info about your visitors’ browser, which OS they use, their system language and more that can be used to learn about your visitor.
Geotargeting, IPs, and language codes can help you get a good idea of where the visitor is located. Targeting a visitor based on their device or language is made possible through modules that detect a mobile vs. a non-mobile browser or extracting language code and geocode to get a location.
But it doesn’t stop there. Convert also detects where the visitor is referred from. This gives us the capability to target visitors from, for instance, Twitter displaying a variation for visitors coming from Twitter. This alone is helpful for launching well targeted campaigns. But what if you want to target users visiting you on the weekend or during the evening? That’s possible as well, plus 100+ audience presets that Convert has available.
Conclusion
Hopefully this article provides you with insight into how Convert works with your website. Most of what I included in this article can be understood from a technological point of view. However, if you read anything that sounds off or doesn’t make sense to you, feel free to leave a comment below.
Related Posts
Originally published July 14, 2015 - Updated September 11, 2020
