MarketingSherpa Blog

Every Research Partner wants a lift, but we know sometimes, those lifts aren’t achievable without learning more about their customers first.

And often, our biggest lifts are associated with radical redesign tests that really shake things up on a landing page. That is because the changes are more drastic than a single-factor A/B test that allows for pinpointing discoveries.

So, how can you strike a balance between using these two approaches while still delivering results that satisfy expectations?

You can achieve this by managing your client’s or business leader’s expectations effectively.

It sounds easier said than done, but there are a few things you can do to satisfy a client’s or business leader’s needs for lifts and learnings. 

Step #1. Start with radical changes that challenge the paradigm

At MECLABS, we often recommend a strategic testing cycle with radical redesign testing (multiple clusters as opposed to a single-factor A/B split) to identify any untapped potential that may exist on a Research Partner’s landing page.

However, you must make sure you are not making random changes to a page to achieve a radically different control and treatment, but are truly focused on challenging the control’s paradigms and assumptions currently being made on the page by testing with a hypothesis.

For example, Sierra Tucson, an addiction and mental health rehabilitation facility, found with a radical redesign from a landing page focused on luxury to a landing page focused on trust resulted better with its target audience. The company also generated 220% more leads with the test to boot.

Step #2. Zoom in on general areas your radical redesign test has identified as having a high potential for impacting conversion

Next, we suggest refining with variable cluster testing, also known as select clusters.

If you identify a radical shift in messaging to be effective, as Sierra Tucson did, you might next want to try different copy, different designs or different offers, just to name a few options.

Step #3. Use A/B split testing to isolate variables and achieve proof of their impact

Lastly, you’ll find iterative testing, also known as single factor testing, as the last remaining best option available for new discoveries. Plus, A/B testing your more generalized findings will ensure they truly make an impact.

So, while one change in tone of copy may have worked better than another, you can specifically split test headlines to isolate which factors you can attribute the better performance.

Why did one headline work better? Was it written better? Or, did it simply tie better into the treatment copy? Or, did the headline have nothing to do with it and that treatment headline only won because the treatment copy was shorter? A/B split testing allows you to isolate variables and know for sure.

Sometimes though, your early tests will not be successful

A common challenge to using a linear approach to testing I have discovered is when initial test results don’t clearly show clients a quick-win or learning. This may cause a client or business leader to quickly lose faith in our process.

An alternative approach to solve this problem would be to develop a test plan that allows for iterative testing or single-factor, A/B split tests along with some radical redesign testing.

There are essentially two ways to approach this depending on your available resources and traffic levels:

1. A test plan that allows for cycles of iterative testing and radical redesign testing in tandem.
2. If you have enough traffic volume and available resources, test a number of iterative treatments for the learnings, while also testing the radical redesigns that have greater potential lifts.

Furthermore, laying out potential advantages and disadvantages to consider when using different testing approaches never hurts, either.

Here are a few trade-offs for each test approach I ask Research Partners to consider first based on their test goals to help us find the right mix for their needs:

Radical redesign tests

Advantages

• Tests are formed using a repeatable heuristic approach that keeps marketing analysis organized around answering a primary research question
• Radical redesigns have shown through our research to generally produce larger relative differences between a control and treatment
• Increased potential for a  lift, but with an equal chance of a huge decline

Disadvantages

• Radical redesigns don’t allow for identifying factor(s) that attribute to change
• Using a rigorous test methodology may be difficult for clients facing barriers to testing
• More time for design analysis and test planning is generally needed for radical redesign tests

Single-factor tests

Advantages

• Some single-factor tests can produce significant results
• Single-factor tests take much less time to plan and develop
• Discoveries about your customers can be directly pinpointed to elements being tested

Disadvantages

• Sites with low traffic using single-factor tests can take longer to validate due to the usually lower conversation rate difference between the control and the treatment
• Lengthy testing cycles can significantly impact other tests in your queue
• A relative change between your control and treatment may not be significant enough to produce a substantial lift

Focus on your client’s or business leader’s goals

Ultimately, finding the right testing mix depends on understanding your client’s or business leader’s goals and making universal test plans to work toward those goals.

This can be a challenge, but keeping the focus on your client’s goals also keeps the focus on the things that matters most – discovering more about their customers and how you can better serve them.

Related Resources:

Competitive Analysis: Stepping outside the industry and ahead of the competition

Who’s Running Your Marketing Department, You or Legal?

Optimization: A discussion about an e-commerce company’s 500% sales increase

Test Planning: 3 simple tips to keep your test planning and optimization on the right track

Categories: Research And Measurement Tags: analytics, conversion rate optimization marketing metrics, marketing insights, marketing strategy, project management, test design, test planning, testing strategy