When revising the design of a website, it is important to experiment with different versions rather than simply constantly changing the look-and-feel of the site.
Making regular changes is good but it's advisable to do it in such a way that you can track the results between the different versions.
If you just go ahead and make a big change, it can be difficult to measure the results as changes in buying habits from one month to the next can influence sales just as much as the changes you might make to the site. This can make it difficult to ascertain whether it was the revisions that made the difference or simply that it's a busier or quieter period than usual.
A/B testing allows you to test different versions at the same time by serving up your different versions randomly to different users.
We are currently running some homepage tests on the Monocle Order (you can see versions A—the original—B, and C in the images above). Our goal here is to see if, B, images of more "real" people will lead to better site interaction and, C, whether removing the emphasis on membership entirely from the homepage will draw more people in.
The interesting thing is that, rather than simply making some big changes and checking back in a month or two to see how they've gone, we're testing all three versions right now in unison. 33% of visitors see each version so we get valuable insight into which ones work better.
It's worth noting that this only works on relatively high-traffic sites as you need a reasonable number of clicks to be able to measure the differences and determine a winning version.
We will post a follow up with some results from this experiment.