Tony Doty

Testing: Go big, or go home?

April 25th, 2013

One of the most common questions and debates we have here at MECLABS is, “How radical do we go?”

Let me explain – for every test, we have an objective we’re trying to accomplish and a set of metrics we’ll use to judge the performance and success of the test. If we “go radical” and change lots of different elements on the page, we might hit it big, or we might tank. But, either way, we wouldn’t know the true impact of any specific change.

If we “go conservative,” we’ll be able to directly tell what the impact of changing a specific element was, ensuring we learn something, but might never be able to hit that lofty conversion goal our team has set.

So, which approach is right? Well, the short answer is they both are. The long answer is the rest of this post.


The right blend between radical and conservative tests

That may sound like a cop out, but a successful test strategy needs to find the right blend between radical and conservative tests. Let’s try an analogy …

Let’s say you just started playing baseball. You’ve had batting practice with your coach and just can’t seem to connect on any pitches. So, your coach starts tweaking. Widen your stance. Lift your elbow. Tilt your head. Tweak, tweak, tweak. But you’re still not hitting anything.

Then, you try something radical. You walk to the other side of the plate and take the first pitch into the outfield. Turns out you bat lefty. That would have been good to know an hour ago. Chances are, you were never going to succeed with small tweaks, because there was something fundamentally wrong with your approach.

The same goes for testing. If you’re making progress with small tweaks, a headline here, button color there, you may never reach your true potential.

We always want to get a solid learning from every test we perform, but looking back through the archives, a lot of the largest wins we’ve ever achieved don’t come from single factorial tests, or variable clusters where we try to focus in on specific elements of the MECLABS Conversion Sequence heuristic like friction or value.

Instead, they come from radical redesigns, where we test a totally new approach or simultaneously improve numerous elements we identified as issues with the page.

Thankfully, based on our methodology, collective experience and past testing results, we normally are able to improve page conversions if we do decide to go radical, but just doing radical redesigns or trying things because they’re “cool” doesn’t equal a sound test plan or long-term testing strategy. Neither is playing it safe 100% of the time.

The trick is to find the right blend of radical and conservative tests. You need to ensure you’re learning about your prospects, their motivations and how best to communicate your unique value proposition.  But, every once in a while, you need to shake things up, throw caution to the wind and try to hit that big home run. If it doesn’t work, that’s fine. At least you know one more thing that doesn’t work. There’s still learning there.

As Thomas Edison allegedly once said, “I have not failed 1,000 times. I have successfully discovered 1,000 ways to NOT make a light bulb.”


Related Resources:

Homepage Optimization: Further test ideas for a page that converts 4 times higher

The WOW! Computer: How firstSTREET applied a radical redesign to deliver a 500% sales increase

Landing Page Optimization: Radical redesign leads to 3,566% increase in conversion

Landing Page Optimization: 3 quick recommendations from the stage at Optimization Summit 2012

Tony Doty

About Tony Doty

Tony Doty, Associate Director of Optimization, MECLABS Tony is responsible for guiding and assisting all aspects of testing for our Research Partners. When not optimizing projects with our Partners, he is guiding other members of the team by providing key insights and training to support their professional development. Prior to MECLABS, Tony was a Product Management Consultant where he performed research to help companies optimize product offerings and marketing strategies. Tony holds a Bachelor of Science in Materials Science from the University of Minnesota and is a self-confessed data junkie. In his spare time, he enjoys playing video games with his wife Ashley, and spending quality time their dog, Barley.

Categories: Marketing Tags: , , , ,

We no longer accept comments on the MarketingSherpa blog, but we'd love to hear what you've learned about customer-first marketing. Send us a Letter to the Editor to share your story.