A/B Testing: One word will unclog your conversion testing
With A/B testing, you’re examining and exploring the mind of the customer. You’re learning about your customers and you’re the one asking the questions. However, the newly released MECLABS Online Testing Course explains in great detail why you can’t ask just any question to get the answers you need.
There’s a formula for what goes into that question, and it’s all built around one imperative word.
Which.
The word “which” demands specifics and precision, allowing you to focus on something that can be answered with a split test.
Let’s expand this further by looking at one of the key principles Flint McGlaughlin, Managing Director, MECLABS, discussed in Session 2 of the course.
- A properly framed research question is a question of “which” and sets out to identify an alternative (treatment) that performs better than the control.
The guiding force of online testing is seeking to better predict the behavior of your customers. To achieve this, you need a research question to tests your hypothesis.
“If your research question is framed wrong, the entire outcome of the test is dubious because you haven’t approached it properly,” Flint said.
Below are some of the examples presented in the course that convey the importance of this essential word.
Not this: What is the best price for product X?
This isn’t specific. The question doesn’t set out particular items to test. “Best price” could be anything.
But this: Which of these three price points is best for product X?
This utilizes the imperative “which.” The implementation of “these three price points” gives you three precise price points to test.
Not this: Why am I losing customers in the last step of my checkout process?
Sure, you may ultimately want to discover why it is you’re losing those customers, but you must start out smaller. This question doesn’t narrow anything down. The last step of the checkout process is quite complicated and there isn’t just one element present.
But this: Eliminating which form element best reduces customer drop-off?
There’s the “which” again. The “form element” is the metric allowing you to compare one specific element to another. This gives you a particular element to test rather than just presenting a broad idea.
Not this: How many objectives should I have on my homepage?
“Many” is much too vague of a term.
But this: Which number of objectives should I have on my homepage?
Ah, a metric. “Number of objectives” moves you toward a more specific, precise research question.
Your A/B testing results can be difficult to unravel, so establishing a clear “which” question from the get-go will help you untie those knots and better understand your customers’ behaviors.
Related Resources:
Web Tracking and Online Testing: Come to the dark side, we have cookies!
Online Testing: 6 test ideas to optimize the value of testimonials on your site
Online testing: Two common reasons to use a radical redesign test approach
Categories: Marketing A/B testing, conversion testing, online testing, online testing course, research question
