Marketing Experimentation: How to determine what to test next
Every Wednesday, we hold a free Marketing LiveClass as part of ChatGPT, CRO and AI: 40 Days to build a MECLABS SuperFunnel. Everyone is welcome to join and learn, as we build marketing funnels with members of the MECLABS SuperFunnel Research Cohort.
In the LiveClass, marketers and entrepreneurs can ask questions in the webinar chat. And we answer them right here…
Next steps – I have been testing big differences – how do we go deeper once we get some success?
This questioner is asking about testing big differences between a control and a treatment in a marketing experiment.
I’ll give you four options to decide what to test next. The option that works best for you will be based on the unique needs and strategy of your business. It should also be informed by what you learned in the original test. That is, after all, the main point of marketing experimentation. To learn about our customers so we can better serve them.
Continuously move through the optimization sequence
The MECLABS Optimization Sequence Heuristic is…
Opr = Optimize Product Factor
Oprn = Optimize Presentation Factor
Ocnn = Optimize Channel Factor
This means you should optimize your product before the presentation of the offer and optimize the channel last.
And if you think about it for a moment, it should make logical sense. If you optimize the channel first, you’re going to spend a lot of money sending traffic to an underperforming landing page presentation of your offer. And if you optimize the presentation first, people will be buying an inferior product that does not have a forceful value proposition.
This is the process we followed in the MECLABS SuperFunnel Research Cohort, without explicitly saying so. While working with the cohort and building their funnel, cohort members were challenged to identify how powerful their product’s value proposition actually is in the competitive marketplace, and how to present the offer. Then they tested in the channel.
I do want to note, this of course is not a stagnant process. Optimization is continual. So once you’ve gotten to the point of optimizing the channel factor and have a profitable flow going, you can circle back to the product and see how to further optimize it.
I also want to note that we break out these factors to help communicate information. In real-world testing, things can get messier and there can be overlap. For example, you can test in the channel to learn about your product.
Test a new place in your funnel
When I think of testing and optimizing a funnel, I think of a classic physical comedian like Charlie Chaplin or Lucille Ball. There is a long rope they need to keep straight, and wherever it droops they need to prop it up. Great, they’ve got it propped up the biggest drooping area. But oh no, now it’s drooping on the other end. Time to prop it up there. I can picture them comically running back and forth.
The same is true for your marketing funnel. You can look at your metrics to determine where to test in your funnel and try to prop up the most drooping area first. Where is your next biggest challenge?
Did you get more people from the landing page into your cart? That’s great. But maybe they are less qualified and now your cart isn’t converting as well. That is your next place to test.
Or maybe you started with the cart or form and improved conversion there. Now it’s time for a test to get more people to that optimized conversion point.
Move to another problem, solution, or tactic
When you first set up your experiment, you likely brainstormed many problems you might want to address, many solutions to those problems, and many tactics you can implement.
Based on what you learned in this test, what problem, solution, or tactic should you test next? There is a nice visual in Marketing Strategy: 4 steps to developing an effective and strategic test that illustrates this concept. When you look at that visual, you can see that you can ‘move over’ to the next problem, solution, or tactic with your next test.
Go deeper
If you tested big differences, you may not know what element of these big differences actually caused the change. For the next test you can go deeper to get an even better understanding of your customer. (Keep in mind though, the downside of this approach is why you tested big differences to begin with – if you get too deep, you may not find a statistically significant difference between the control and treatment).
For example, let’s say you have a shampoo bar. Maybe you tested the environmental friendliness of the shampoo bar versus the level of cleanliness the shampoo bar provides.
Let’s say, environmental friendliness won. But what aspect of environmental friendliness? Was it the lack of chemicals? Or less wasteful packaging?
Another way to go deeper is to test different expressions of the winning treatment. The winning treatment helped you identify an essence that was more effective with the customer – environmental friendliness. Now, you can test which expressions work best – which headlines about environmental friendliness, which images, which CTAs, etc?
Here’s one more way to go deeper – zoom into subsets of customers. You found the generally most appealing offer to the broadest set of potential customers. Now segment by, let’s say, US customers versus European customers. Start with your highest priority/largest segment and go from there. Which offer is most appealing to them? Which expression of the offer? And on and on.
Plan ahead for your testing
The above approaches don’t only have to happen after the test has run its course. In fact, you can consider many different tests when you begin, and then set a sequence for what to test next based on the results of your tests. ‘If the control wins, that means X about the customer, so next I will test…’ ‘If the treatment wins, that means Y about the customer, so next I will test…’
To help you prioritize your tests, you can use our free Test Planning Scenario Tool.
Are the MY’s constantly being optimized based on efficiency and conversion?
The MY’s are the micro-yeses, the sequence that your customers go through when making a decision about your conversion objective. You can see a discussion of the micro-yeses in Website Wireframes: 8 psychological elements that impact marketing conversion rates.
And to build on the previous question, yes, you should constantly optimize how you handle the micro-yes sequence throughout your funnel and on your landing page, based on what you are learning from your tests and other ways you are measuring customer behavior.
Essentially, you should ask – ‘am I serving potential customers, and the questions they will consider on their micro-yes journey?’ Your tests will help you answer that question and conversion rate performance will as well.
But also listen to their feedback directly to help you understand what you are seeing in the numbers. Talk to them. Give them an easy way for them to contact you. That qualitative information will help you make sense of your conversion and testing data.
This is where followup testing can help as well, and you can always bring something you learned somewhere else in the funnel back to your core – your landing page (I discuss the centrality of the landing page in Conversion Marketing and Landing Page Optimization: Don’t overlook the center of your marketing investment).
Learn something interesting in a channel? How does that affect your landing page, and what followup tests will you run on the landing page to get the most juice for the squeeze?
Am I a free guest here watching a course that these folks paid for? How did they get access to this Q/A database?
You can RSVP now to be a free guest and learn from a Marketing LiveClass on Wednesday at 4 pm EDT as well. Here are some excerpts from recent LiveClasses to give you an idea of what you can expect…
Probably the most frightening thing I have seen in AI development
3 things you can do if your test data is not actionable
What does a good offer look like?
