It’s all good that I’ve started playing with the UI of this blog. But, I did like to know what works, and what doesn’t. Google Analytics gives me metrics that let me know what’s being read. Inspectlet gives me how this blog/site is used, what do my readers do, while on this blog. So, the question now becomes… How do I measure the effectiveness of the UI changes I make to this blog? What are the metrics?
To answer those two questions, I would have to dive in, and think about what do I consider as goals? What do I want my readers to do on this blog? Then, I need to find the best UI changes that make attaining those goals possible.
To do that, I’d have to start with A/B, & Multivariate testing. And so, this post is exactly about that!
What’s A/B & Multivariate Testing?
A/B testing, or split testing is essentially comparing two versions of a web page, or in the case of this blog - a post, to see which of the versions retain readers for a longer time on this blog. Provided the fact that retaining the readers for a longer duration, on this blog, is a goal.
Multivariate testing is essentially the same as A/B testing but it’s more granular. Which allows you to test, and compare more variables, thus revealing more information about how the variables on a particular A/B test page/post add to achieving the above mentioned goals.
There are many tools out there that let you do this. The best, and most simple ones are paid, and are used daily by a lot of businesses to fine tune their conversion rates, sign-up rates, etc. A/B, and Multivariate testing is also used on mobile applications to test features, and funnels.
This blog on the other hand, does not have much in terms of conversation rates. The UI changes I am working on are more of an experiment than anything else, meant to teach me more about UI, UX, design, and A/B testing in general.
With that being said, I don’t want to spend on a monthly subscription for carrying out experiments with this blog (yea, I am a cheap-skate that way). In that regards I set out to hunt for free A/B, and Multivariate testing tools that could work with a Jekyll blog. Below is the list of what I found, and below the list of tools/frameworks there’s another list of resources, and articles that I thought worth a mention here.
Another thing to note here would be that this blog is statically generated with Jekyll. This limits my options further, but surprisingly there are plenty of A/B, and Multivariate testing tools to choose from. Some really old, others more recent.
8 Free A/B Testing Tools That Will Work With Jekyll
- SixPack: One of the good ones out there. This A/B testing framework is language-agnostic. You can use it with any language. What I like about SixPack is that it includes a very nicely designed dashboard for viewing the results, and making decisions. To be able to use SixPack across multiple web services, written in any language, you would have to first install the SixPack framework which is the Sixpack-server. The Sixpack-server receives requests from your web services or sixpack-client pages (multiple client libraries are provided). There’s a third component - Sixpack-web, which is the dashboard included in the framework. It’s a full-fletched framework, and it comes awesomely packed with a web interface. It’s updated frequently. What’s holding me back from using SixPack is the fact that the setup will take it’s own sweet time. If I was testing conversions for a e-commerce website, or sign-ups for a web-app in production, etc. I wouldn’t blink twice before using SixPack. But since it’s a static blog we’re talking about, I am gonna give SixPack a pass.
- Clutch.io: Clutch.io is from Twitter. Or rather was. It’s old, and hasn’t been updated in a long time. None the less it’s still out there, and it can primarily be used for native A/B testing your iOS or Android Apps. There are two parts to clutch.io; a) Native A/B testing for iOS and Android apps. b) Toolkit for testing hybrid native/HTML apps for iOS. And since this blog is not an app, we shall not use Clutch.io for that reason. I was under the impression that Clutch.io had a component for web app testing too, but alas I made a mistake. Thus technically, it’s seven free A/B testing tools!
Lessons Learned from 100,000 A/B Tests
Here’s a not-so-long video about the importance of A/B testing on the TNW channel on youtube. Dan Siroker is the presenter at the TNW Conf. Europe 2015.
A/B Testing Resources, Articles, & Posts Worth A Mention
- The Complete Guide to A/B Testing - vwo.
- Comparing a Multivariate Test to an A/B Test - optimizely.
- 20 Lines of Code that Will Beat A/B Testing Every Time - Steve Hanov.
- Mobile A/B Testing Made Easy - Socialcam.
- A/B Testing for Beginners: 70 Resources to Get You Started - Quicksprout.com
- GoodUI.org/blog - I wrote a post about GoodUI.org sometime back…well just recently they launched a blog! I would highly recommend subscribbing via RSS.
Expect more posts on A/B testing, UI, UX, and the design of this blog. For now I am still torn between Abba, and easyAB.js.
Hope this helps someone!