Testing: Go big, or go home?
One of the most common questions and debates we have here at MECLABS is, “How radical do we go?”
Let me explain – for every test, we have an objective we’re trying to accomplish and a set of metrics we’ll use to judge the performance and success of the test. If we “go radical” and change lots of different elements on the page, we might hit it big, or we might tank. But, either way, we wouldn’t know the true impact of any specific change.
If we “go conservative,” we’ll be able to directly tell what the impact of changing a specific element was, ensuring we learn something, but might never be able to hit that lofty conversion goal our team has set.
So, which approach is right? Well, the short answer is they both are. The long answer is the rest of this post.
The right blend between radical and conservative tests
That may sound like a cop out, but a successful test strategy needs to find the right blend between radical and conservative tests. Let’s try an analogy …
Let’s say you just started playing baseball. You’ve had batting practice with your coach and just can’t seem to connect on any pitches. So, your coach starts tweaking. Widen your stance. Lift your elbow. Tilt your head. Tweak, tweak, tweak. But you’re still not hitting anything.
Then, you try something radical. You walk to the other side of the plate and take the first pitch into the outfield. Turns out you bat lefty. That would have been good to know an hour ago. Chances are, you were never going to succeed with small tweaks, because there was something fundamentally wrong with your approach.
The same goes for testing. If you’re making progress with small tweaks, a headline here, button color there, you may never reach your true potential.
We always want to get a solid learning from every test we perform, but looking back through the archives, a lot of the largest wins we’ve ever achieved don’t come from single factorial tests, or variable clusters where we try to focus in on specific elements of the MECLABS Conversion Sequence heuristic like friction or value.
Instead, they come from radical redesigns, where we test a totally new approach or simultaneously improve numerous elements we identified as issues with the page.
