Split Testing: The Top 3 A/B Testing Mistakes Businesses Make All The Time
The following article is a Guest post from Aaron Gray! Aaron is the co-founder of Studio 56 and is a passionate digital marketing expert who has worked with some of the largest digital marketing agencies in Australia.
He has been working in the digital marketing field for ten years. Aaron loves to travel the world to not only enhance his cultural experiences but learn and enhance his skills in the digital marketing industry.
He is dedicated to helping others reach their online marketing goals. Take it away Aaron!
Split testing (better known as A/B testing) is a great way to measure your current marketing efforts and to see whether changes to any section of the website can dramatically improve your conversions and leads.
Split testing while a great way to gain more insight into your website and to understand your overall audience better, can unfortunately get mixed results when businesses don’t understand the concept of A/B testing.
When this occurs businesses can make some common mistakes that lead to inaccurate data. The following are the common mistakes that many businesses make when A/B testing that should be avoided.
The Top 3 Split Testing Mistakes Businesses Make
Split Testing Mistake #1: Split Tests Aren’t Run For The Required Length Of Time
When running A/B tests many businesses make the mistake of cutting their test short after a few days.
This unfortunately won’t give you the results you’re looking for. Instead it’s ideal to run your test for up to 7 days or until you’re 95% sure of the winning version of your website or page.
You could also test until you have reached at least 100 conversions. If you decide to test through your conversions, it’s important to make sure you keep track of how fast you reached those 100 conversions. This will give you valuable insight into which version of the website your visitors like the most.
Did you know that More than 20 percent of businesses have reported that they do not have an effective strategy for landing page testing? (Source: IonInteractive)
So why do business cut their A/B testing short? Generally, it’s because it’s difficult to wait and when you see a large spike in conversions you become 97% sure which is the winning version.
The problem with this is that it doesn’t necessarily give you accurate data to fall back on. It’s not unheard of that in the first couple of days one version overtakes the other to only fall short by the end of the week with the second version winning overall.
This is why it’s important to wait and to gain as much data as possible to get the best results. So basic rule of A/B testing length includes:
- Running the test for at least 7 days. If there’s a close margin run it for another 7 days.
- Never change any other elements on the website during the testing time as this will give mixed results.
- Only run one test at a time during the 7 day period as multiple tests may cause problems with cross traffic and insufficient data results.
Split Testing Mistake #2: Not Paying Attention To External Factors Offline
Throughout the year different occasions, events, and seasons can dramatically influence what your customers like and don’t like. Many businesses make the mistake of not paying attention to external factors that reside offline which can affect their A/B testing.
As an example if you’ve A/B tested during Christmas and continue with that version during January and February, you may see different results to what you first experienced during the holiday season.
Split split testing concept. Marketing or SEO specialist select better result in A/B split testing. Wide banner composition with bokeh in background.
Split testing isn’t a once off, it needs to be continuously addressed and used throughout the year to maximize your conversions.
A certain version of your website may work best during the winter months, but won’t work during the summer months.
This is why it’s imperative to A/B test around major seasonal changes, events, or occasions that can help you to determine whether your current website version is still working strong. Some tips when A/B testing for external factors include:
- Test in the weeks leading up to the event, season, occasion, and the weeks after to see the difference in what visitors like to see.
- When creating the second version of your website or page to mimic the external factor, try to replicate some aspect of it in your design to make it relatable for visitors. This can help to boost conversions.
- A/B test for 7 to 14 days depending on what your results are like after the first 7 days.
Split Testing Mistake #3: Testing Without A Hypothesis
The idea of A/B testing is to create a hypothesis and test to see whether you receive the desired results based on the hypothesis. Unfortunately, there are some businesses which A/B test randomly without having a clear goal or hypothesis in mind.
This can lead to a huge expense which can waste time and traffic. Instead you need to setup a hypothesis before you test. A hypothesis consists of three main elements. These include:
- The variable (if)
- The desired result (then)
- Rationale (because)
These three areas should be researched, drafted, and documented prior to completing a live A/B test. Let’s look at them in more detail.
The Variable
The variable is the element of a website which can be modified, added or removed to produce an outcome that’s desirable. Some of the best things to A/B test include:
- Call to action
- Headlines
- Layout and style of website
- Text
- Navigation layout
The Desired Result
The desired result is the predicted outcome that you’re hoping to achieve. This could be more clicks, conversions, or taps on the button etc.
The result will be measured based upon what you’re measuring it against. Do you believe the change will make a large-scale effect or produce incremental difference to the website?
Rationale
Rationale refers to the reason why you’ve done your A/B hypothesis and the research that was conducted to make the changes. It gives you the motivate as to why you decided to A/B test your website in the first place.
What’s The Overall Results Of Your Testing?
The whole idea of a hypothesis is to gain insight into whether changing certain areas can help to boost your conversion rate. Without creating a hypothesis to base your A/B test on you will blindly change areas of your website without gaining any real results.
Overall
A/B testing when done right can be a great asset to any business when trying to boost conversions. By knowing the mistakes that so many different businesses make when A/B testing you can easily change your tactics to make sure your A/B testing is a success and profitable one.
So, I would love to know! Are you making these mistakes when conducting your split testing? Drop a line below to tell me.
Thanks!
