Over the past few weeks I have come across many useful online resources related to multivariate testing and now the time has come to put together a little Top 10 list. This is recommended reading/viewing for all e-marketers, web developers and testers, newbies and pros alike, and it covers the subject from many different angles.
So, here we go!
TOP 10 RESOURCES ON MULTIVARIATE TESTING
1. Experimentation and Testing: A Primer
A good introduction to the various testing methods by Avinash Kaushik in his widely read blog Occam’s Razor. Avinash provides pros and cons of A/B Split Testing, Multivariate Testing and Experience Testing (Avinash’s own term).
Billy’s blog is dedicated to multivariate testing and landing page optimization based on his experience at Widemile. Here is a selection of his posts which explain the basics of multivariate testing in more detail (and graphically for better clarity), provide some testing terminology and explain the difference between Full and Fractional Factorial Test Designs.
Jonathan Mendez’s great contribution – his guide to multivariate testing in six chapters. Includes the following parts: Introduction to Factorial Designs, Choosing a Test Array, Strategy & Tactics for Element Selection, Creating Element Variations, Test Set-Up & Implementation and Understanding Result Data.
The ‘squirrel’ people have put together this fantastic list of 108 (yes, 108!) ideas of improving your conversion rate. Many of these are directly related to testing, so this is also essential reading for anyone involved in MVT.
This is a great resource helping to come up with ideas for testing – which elements of the website to test and in what way. Relevant for all multivariate tests, not just using Google’s Optimizer.
A good checklist of important things to remember when carrying out multivariate testing – again, relevant not just for Google Optimizer, but for any other tool.
Interesting statistics presenting the most common barriers to implementing multivariate testing.
As mentioned in one of the earlier posts, Google released The Techie Guide to Google Website Optimizer. The 26 page document, takes a comprehensive look at the Optimizer tool from a more techie angle and is well worth a look.
It’s good to keep an eye on what the Google team are up to in their multivariate testing efforts…
Series of webinars led by Bryan Eisenberg of FutureNow along with Tom Leung, Product Manager for Google Website Optimizer. Bryan is the author of Always Be Testing and is a highly regarded specialist in website testing and online marketing. The webinars contain great commentary and present many visual examples and case studies of tests. The series consists of the following presentations: