The Oban Blog

Multivariate Testing – Top 10 Online Resources

Over the past few weeks I have come across many useful online resources related to multivariate testing and now the time has come to put together a little Top 10 list. This is recommended reading/viewing for all e-marketers, web developers and testers, newbies and pros alike, and it covers the subject from many different angles.

1. Experimentation and Testing: A Primer

A good introduction to the various testing methods by Avinash Kaushik in his widely read blog Occam’s Razor. Avinash provides pros and cons of A/B Split Testing, Multivariate Testing and Experience Testing (Avinash’s own term).

2. Breaking down multivariate testing (Billy’s Optimization Guide)

Billy’s blog is dedicated to multivariate testing and landing page optimization based on his experience at Widemile. Here is a selection of his posts which explain the basics of multivariate testing in more detail (and graphically for better clarity), provide some testing terminology and explain the difference between Full and Fractional Factorial Test Designs.

3. Expert Guide to Multivariate Testing Success

Jonathan Mendez’s great contribution – his guide to multivariate testing in six chapters. Includes the following parts: Introduction to Factorial Designs, Choosing a Test Array, Strategy & Tactics for Element Selection, Creating Element Variations, Test Set-Up & Implementation and Understanding Result Data.

4. 108 Ways to Improve Your Website’s Profits

The ‘squirrel’ people have put together this fantastic list of 108 (yes, 108!) ideas of improving your conversion rate. Many of these are directly related to testing, so this is also essential reading for anyone involved in MVT.

5. 64 Tips for Getting Started with Google Website Optimizer

This is a great resource helping to come up with ideas for testing – which elements of the website to test and in what way. Relevant for all multivariate tests, not just using Google’s Optimizer.

6. Top 15 To-Do List for Multivariate Testing with Google Website Optimizer

A good checklist of important things to remember when carrying out multivariate testing – again, relevant not just for Google Optimizer, but for any other tool.

7. Multivariate Testing is Low Hanging Fruit for Marketers

Interesting statistics presenting the most common barriers to implementing multivariate testing.

8. Google Website Optimizer Techie Guide

As mentioned in one of the earlier posts, Google released The Techie Guide to Google Website Optimizer. The 26 page document, takes a comprehensive look at the Optimizer tool from a more techie angle and is well worth a look.

9. The Official Google Website Optimizer Blog

It’s good to keep an eye on what the Google team are up to in their multivariate testing efforts…

10. Always Be Testing Webinars

Series of webinars led by Bryan Eisenberg of FutureNow along with Tom Leung, Product Manager for Google Website Optimizer. Bryan is the author of Always Be Testing and is a highly regarded specialist in website testing and online marketing. The webinars contain great commentary and present many visual examples and case studies of tests. The series consists of the following presentations:

  • Landing Page Design and Layout
  • Confidence and Trust Building
  • Testing Headlines and Calls to Action