A/B testing, also known as split testing, is an important aspect of any research into conversion rate optimisation. Essentially, A/B testing is when the process of comparing two versions of a web page against each other in a live environment. We use this method in order to secure a measurable way of tracking whether a change to a site has performed better than the original.
By displaying two variations of your campaign, you can see which one attracts more interaction and conversions from your customers. An alternative to this approach is simply putting the change live, known as a direct change, which is a lot more difficult to track.
There are two key considerations to bear in mind: sample size and statistical significance. Any tests should be run until they have reached a valid sample size, and until statistical significance is reached. Sample size is important in order for your results to be representative. Statistical significance minimises the risk of coincidence or fluke.
You don’t want to make a change to a website unless you know the improvement you are seeing is statistically valid. Your goal is to develop a good hypothesis which is based on data and research. Without validated data, you’re reliant on a hunch.
Designing test-based hunches introduces all sorts of bias, which may not solve any real problems on the site and can be a huge waste of time. We used data-driven A/B testing to increase a client’s conversion rate by 132%, showing how valuable this method is when done correctly.
How A/B testing grows your business
A/B testing can make a vast difference in how effective your marketing efforts prove to be, hence saving you time and making you more money in the long run. You need to ensure the branded message you send out to your potential clients fits not only your small business’s image but will also draw in customers. With A/B testing you can easily hone in on the most effective elements of your marketing piece to ensure your efforts are not in vain!
The reason why A/B testing is so important is that it can make a dramatic difference to your bottom line. It has been observed that even minute alterations in emphasis, wording, layout or even visual elements can produce significant improvements in the receptivity of audiences. That is why A/B testing is crucial; until you run tests you will not know the impact of these minute changes.
If you use tightly controlled tests you can gather empirical data to ascertain which elements will function optimally to attract clients to your small business and the products and services you offer. Although best practice modalities may sound good, A/B testing offers you real, raw data that will allow you to properly understand your audience. Sometimes conventional marketing wisdom just cannot predict how people will react to material.
A/B testing is crucial to gain an understanding of conversion rates. You can gauge this, for example, by testing a call-to-action button, this will allow you to understand how much better one page may work than another for engaging potential clients.
Once you have figured out that one variation is significantly more successful than any other, drawing more audience attention which translates into more profit for your small business, then you will realize that to post marketing pieces without A/B testing is foolhardy.
In turn the data you glean will guarantee that in the future it is easier to create marketing pages. However even if you have identified successful elements it is important to continue to test regularly as what was previously effective may change with time.
A/B testing is also crucial to guarantee that your desktop version of a page is as successful as the mobile version. Your small business may have set up two fairly different landing pages however as mentioned above, minute changes may affect the effectiveness of your page and how attentively your potential customers engage in your marketing material.
Due to the different formatting options that PC and mobile offer you will need to have different pages, however you need to ensure that both will work efficiently. By A/B testing both versions separately you can ensure that one does not outperform the other and that both work consistently to convert audiences into customers. In some cases, making major revisions to your site can result in considerable costs or significant strategy changes.
A/B testing can help you examine visitor and customer behavior on your site before committing to major decisions and help you increase your chances of success. In short, A/B testing helps you avoid unnecessary risks by allowing you to target your resources for maximum effect and efficiency, which helps increase ROI whether it be based on short-term conversions, long-term customer loyalty or other important metrics.
External factors can affect the results of your test. Be sure to factor in holidays, force majeure and any other occurrences which can impact customer behavior when planning your campaigns and tests.
Useful tips to keep in mind
Here are a few tips we’ve picked up from our testing:
- Get the big picture figured out. If you don’t have your messaging down, no amount of testing will show big gains.
- Implement tests quickly with Optimizely’s visual editor to save time. Do as little development as possible before getting a test up and running.
- Test big changes, not button colors. Try different messaging, new headlines, be bold.
- Always have a test running.
- Keep a larger perspective when designing tests.
A/B testing is a double-edged sword.
On one hand, being a data-driven company is what we should all be striving for. A/B testing gives you hard data, which helps you make confident decisions. We’ve seen some decent wins in our own testing with very positive results.
On the other hand, data without context can cause you to make poor decisions. For example, let’s say I run an A/B test where I show 50% of my visitors a coupon, and test for purchase conversions. Obviously the customers that received the coupon will convert higher, but does that really help me in the long run?
That’s a really obvious example, but there are much more subtle ways in which A/B testing can be deceptive. Let’s say you make a change to hard-sell your customers using slimy marketing tricks, which results in a higher conversion rate. That may win an A/B test, but damage your brand in the long run.
When zooming in on details like “does a blue or a red purchase button convert better?”, it’s important to keep perspective on your larger business goals.