Marketingsherpa Blog

In a recent blog post for our sister company MarketingExperiments, I shared my experiences at the fifth Design for Conversion Conference (DfC) in New York City. Today, I want to focus on a topic from Dr. Dan Goldstein’s presentation, and its relevance to usability and product testing for marketers — how focus group studies can effectively misrepresent true consumer preferences.

Asking you for your input on our Landing Page Optimization survey for the 2011 Benchmark Report has firmly planted the topic of surveys at the forefront of my thinking.

Calibration is not the whole story

The need to calibrate focus group data is well recognized by marketers and social scientists alike. The things marketers want to know the most – such as “intent to purchase” – is more obviously susceptible to misleading results. It’s easy to imagine that when people are asked what they would do with their money in a hypothetical situation (especially when the product itself is not yet available), naturally their answers are not always going to represent actual behavior when they do face the opportunity to buy.

However, mere calibration (which is a difficult task, requiring past studies on similar customer segments, where you can compare survey responses to real behavior) is not enough to consider. How we ask the question can influence not only the answer, but also the subsequent behavior, about which the respondent is surveyed.

Dr. Goldstein pointed me to an article in Psychology Today by Art Markman, about research into how “asking kids whether they plan to use drugs in the near future might make them more likely to use drugs in the near future.” Markman recommends that parents must pay attention to when such surveys are taken, and make sure that they talk to their children both before and after to ensure that the “question-behavior effect” does not make them more likely to engage in the behaviors highlighted in the surveys. The assumption is that if the respondent is aware of the question-behavior effect, the effect is less likely to work.

Question-Behavior Effect: The bad

If your marketing survey is focused on features that your product or service does not have—whether your competitors do or do not—then asking these negative questions may predispose your respondents against your product, without them even being aware of the suggestion. This is especially worrisome when you survey existing or past customers, or your prospects, about product improvements. Since you will be pointing out to them things that are wrong or missing, you run a good chance of decreasing their lifetime value (or lead quality, as the case may be).

Perhaps the survey taker should spend a little extra time explaining the question-behavior effect to the respondent before the interaction ends, also making sure that they discuss the product’s advantages and successes at the end of the survey. In short, end on a positive.

Question-Behavior Effect: The good

However, there is also a unique opportunity offered by the question-behavior effect: by asking the right questions, you can also elicit the behavior you want. This means being able to turn any touch point—especially an interactive one like a customer service call—into an influence opportunity.

I use the word “influence” intentionally. Dr. Goldstein pointed me to examples on commitment and consistency from Robert Cialdini’s book Influence: Science and Practice, such as a 1968 study conducted on people at the racetrack who became more confident about their horses’ chance of winning after placing their bets. Never mind how these researchers measured confidence—there are plenty of examples in the world of sales that support the same behavioral pattern.

“Once we make a choice or take a stand, we will [tend to] behave consistently with that commitment,” Cialdini writes. We want to feel justified in our decision. Back in college, when I studied International Relations, we called it “you stand where you sit”—the notion that an individual will adopt the politics and opinions of the office to which they are appointed.

So how does this apply to marketing? You need to examine all touch points between your company and your customers (or your audience), and make a deliberate effort to inject influence into these interactions. This doesn’t mean you should manipulate your customers—but it does mean that you shouldn’t miss an opportunity to remind them why you are the right choice. And if you’re taking a survey—remember that your questions can reshape the respondents’ behaviors.

P.S. From personal experience, do you think being asked a question has influenced your subsequent behavior? Please leave a comment below to share!

Related Resources

MarketingSherpa Landing Page Optimization Survey

Focus Groups Vs. Reality: Would you buy a product that doesn’t exist with pretend money you don’t have?

Marketing Research: Cold, hard cash versus focus groups

Marketing Research and Surveys: There are no secrets to online marketing success in this blog post

MarketingSherpa Members Library — Are Surveys Misleading? 7 Questions for Better Market Research

After squinting at my screen for weeks trying to read the MarketingSherpa 2011 Email Marketing Benchmark Report PDF, I finally have a hard copy sitting on my desk — and it’s bursting with insight.

Having read the executive summary weeks earlier, I flipped through the chapters today and was struck by this stat:

Does your organization have a method for quantifying ROI from email marketing?

• No: 59%
• Yes: 41%

Email marketing can be amazingly efficient. B2C marketers report an average 256% ROI from the channel — pulling in $256 for every $1 invested — as mentioned later in the report.

What shocks me is that 59% of email marketers have not gauged their program’s efficiency. This means their company executives are likely unaware of the amazing job they’re doing. Even if executives have seen the clickthrough and conversion rates, they’re likely thinking about that line from Jerry Maguire.

Show me the money

At last week’s Email Marketing Summit, Jeanne Jennings, Independent Consultant and MarketingSherpa Trainer, shot holes in many of the excuses she’s heard for why companies can’t calculate email’s ROI.

Here are three she highlighted:

1. Our Web analytics software doesn’t provide this information
2. We can’t track online sales back to email
3. We don’t have an exact figure for costs

Taking these one at a time, Jennings noted that 1) most analytics solutions can provide the information. Google Analytics does and it’s free. 2) Setting up the tracking is simple. 3) You don’t need exact figures.

“As long as you can compare in an apples-to-apples fashion, that’s enough to get started,” Jennings said.

Judging performance by clickthrough and conversion rates is not enough — you should know the revenue generated, both on a campaign-level and a broader program-level.

Two simple calculations Jennings suggested:

• Return on investment: Net revenue / cost

• Revenue per email sent: Net revenue / # of emails sent

On a campaign-level, these metrics will reveal which campaigns pull in more money — not just more clicks. For your overall program, they quickly convey the importance of your work.

Also: The movers and shakers in your company are going to be much more impressed with figures that include dollar signs.

Show email’s potential

Another way to convince executives of email’s power is to point to success at other companies. Also at the Email Summit last week, Jeff Rohrs, VP, Marketing, ExactTarget, mentioned Groupon as a great example that email marketers could rally around.

Forbes recently dubbed the localized deal-of the-day website the fastest growing company ever, and its success is largely due to great email marketing.

The Wall Street Journal mentioned Groupon’s 50 million email subscribers as a competitive advantage and that some analysts estimate its value at $15 billion.

The executives will care

Once you can clearly attribute revenue and ROI to email, you might be surprised at how much attention you attract from company leaders.

At the Email Summit, Philippe Dore, Senior Director, Digital Marketing, ATP World Tour, presented his team’s email strategy to sell tickets to professional tennis events. A single email drove over $1 million in revenue, and several others brought in over $100,000 each.

The overall email campaign generated about $1.5 million in total. Suddenly, ATP’s executives were interested.

“We have our CMO talking about email marketing and subject lines,” Dore said.

Related resources

Email Marketing Summit 2011: 7 Takeaways to improve results

Email Marketing Awards 2011 Winners Gallery: Top campaigns and best results

Live Optimization with Dr. Flint McGlaughlin at Email Summit 2011

MarketingSherpa 2011 Email Marketing Benchmark Report

MarketingSherpa Email Essentials Workshop Training with Jeanne Jennings

Photo by: SqueakyMarmot

“The best research is when individuals pull out their wallet and vote with cold, hard cash.” – my first boss

My first experience in marketing was working with a specialized publishing company. I had the privilege to work on exciting products with sexy topics such as “human resource compliance regulations.” Trust me when I tell you there is no better ice-breaker at a party than talking about a ground-breaking court ruling that will change how your company meets compliance of the Fair Labor Standards Act (FLSA).

As a publisher, we used direct-response marketing to drive sales, with an aggressive program of direct-mail, email and telemarketing. And when it came to new product development, we were big believers in research. From customer surveys to industry research to focus groups, we used it all to make the best possible decision. At least, that was the general assumption…

Out of focus

You always have to test because many research tactics just help you achieve a best guess. And while a best guess is often closer to the truth than a random guess, it’s sometimes widely off the mark. In fact, I learned a valuable lesson one day when our company performed a focus group.

The members of this particular focus group were subscribers of a paid newsletter, and we knew that each person had subscribed by responding to a specific direct mail piece. That mail piece was extremely effective, with a powerful but somewhat provocative subject line and letter. Many people loved that direct-mail piece, but many hated it, so we wanted to get the opinion of the focus group members. When we showed the group the direct-mail piece and asked them if they would respond to that piece, 40 percent said they would never respond (if they only knew what we knew). Wow, we were shocked!

So, should we conclude that those 40% were bold-faced liars? Not necessarily. What we can conclude is that what people say they will do and what they actually do may be totally different. That is why research is only part of the equation, but if you want to sleep well at night, you have to take the next step…

Voting with their wallets

At the end of the day, the best research was when we tested the product and let the customers in the marketplace determine with their wallet if it was a viable product. We would test critical elements, like book title and price, and very quickly we would know if we had a winner or not.

Yes, all of the surveys and research were necessary to get started, but the most critical research was in our testing program. Testing is an amazing research tool. Regardless of the conversion you are trying to achieve, when your prospect takes (or doesn’t take) an action, you a have a valuable piece of information. Your conversion goal may be an event ticket sale, a white paper download, an email newsletter signup, or hundreds of other possible actions, but one thing never changes – the action you are seeking to drive can be tracked.

And if you’re ready to measure when your prospect engages with you, that is when the learning begins.

So, I’m thankful for that boss early in my career telling me repeatedly that the best research is when individuals pull out their wallet and vote with cold-hard cash. Over the years, I’ve had many experiences when individuals tell me they are going to do something but until they actually do it, I’m a little skeptical. (Editor’s Note: It’s true. Todd told me he was going to write a blog post for quite awhile. Now, I believe it.)

So gather as much research as possible, but always remember that cold, hard cash is a pretty sweet piece of research.

Related resources

Are Surveys Misleading? 7 Questions for Better Market Research (Members Library)

Marketing Research and Surveys: There are no secrets to online marketing success in this blog post

Focus Groups Vs. Reality: Would you buy a product that doesn’t exist with pretend money you don’t have?

Never Pull Sofa Duty Again: Stop guessing what your audience wants and start asking

A new report from ExactTarget and CoTweet reveals interesting differences in consumers’ motivations and habits when connecting with brands via email and social media. Take a look to find out more about why your customers are listening to you.

The report (you’ll be asked for an email address and phone number) is the result of three types of focus groups conducted with 44 people, and a 1,506-person survey (see methodology). It is loaded with interesting stats, such as:
o 38% of U.S. online consumers are fans of a brand on Facebook
o 5% follow at least one brand on Twitter
o 93% receive at least one permission-based email per day

The report offers plenty of other great metrics, and touches on useful topics, such as the motivational differences between consumers who first check email in the morning and consumers who first check Facebook. Check out the report for more.

The report also offers great best-practice advice for communicating with customers via social channels. However, there was one piece of advice I want to offer a different opinion on.

The report suggests that marketers avoid promoting exclusive, channel-specific offers in social media, and that “tone and content should be the primary differentiators in our channel strategies, not promotions.”

In general, this is sound advice. Integrating campaigns through multiple channels always drives stronger performance. And you do not want to condition followers to receiving special deals.

However, I feel like marketers should throw their social media followers an occasional treat. They are often truly fans of your brand. I do not think it could hurt to make them feel special, say, once every six months.

The “treat” does not have to be a discount or offer, either. For example, it could be a hint of a product launch sent to the audience two days before a press release is issued. And if it is a deal, it does not have to be exclusive to social media followers. Maybe, just once, they receive a coupon code a few days before your email subscribers.

How do you feel about occasionally giving social followers and fans special treatment? Waste of time? Vital display of gratitude? Let us know…

Many marketers have yet to explore mobile webpages as a marketing opportunity. If you are putting off tests because you think they are too complicated, take a look at this case study on mobile page testing we published today.

Mike Brown, VP, Internet Optimization, Vegas.com, and his team tested if catering to a mobile audience could improve site performance. The team showed mobile visitors a homepage and category pages designed for their devices. The test was a proof-of-concept on whether to invest in an end-to-end mobile experience for Vegas.com.

The tests successfully improved site metrics such as bounce rate and conversion rate (see the case study for details), and Brown’s team plans to rollout more mobile pages this fall.

Brown said these tests were easier than his team anticipated. From our conversation, I understood two criteria the team met to successfully run them:

- Mobile traffic present

Testing mobile pages only makes sense if your website receives mobile traffic. The team looked at its site analytics and noticed mobile visitors accounted for 7% of traffic, and growing rapidly.

- Testing and page design expertise

The team had extensive testing experience with Brown as the head of Internet Optimization. Also, the team used SiteSpect‘s multivariate and A/B testing tool to run tests. SiteSpect team members helped Brown’s team learn to create mobile pages, which Brown noted was surprisingly easy.

So if your team has capable developers, some mobile traffic and good testing experience, there should be little preventing you from running similar tests. And remember: you don’t have to dive in with a full investment. Brown’s team only dipped its toe and is now comfortably moving forward.

There is an array of webpage testing solutions available, helping marketers improve their landing pages, homepages and other online real estate. With so many options, it’s easy to get wrapped up in selecting tools, setting them up and testing them.

The problem is some marketers will spend months selecting and deploying a tool only to A/B test button colors, or different images on the same layout, says Lance Loveday, CEO, Closed Loop Marketing.

“To me, that feels like running 25 miles of a marathon and walking the last one.”

Time is much better invested in researching page data and designing tests that have the strongest likelihood of success, Loveday says.

“90% of your time should be in the planning and actual analysis and coming up with insights, and 10% should be in the technology.”

We spoke with Loveday for an upcoming MarketingSherpa article on how to select better landing page tests. One key to Loveday’s strategy is gathering thorough research, including:
o Analytics data
o Click-tracking analysis
o Qualitative usability studies
o Expert reviews

“We try to marry up quantitative analytics data with qualitative user experience and user profile information to develop some hypotheses for what the problem areas [on a page] might be,” Loveday says.

By digging through this information, your team can identify areas for improvement, attempt to diagnose problems and test solutions. Furthermore, you can estimate tests’ potential impact and prioritize those expected to bring the greatest benefit. Keep an eye on our Great Minds newsletter for more information.

People who follow your social media updates are likely fans of your brand. Their motivations may vary, but if they’re reading and responding to your content, then they know who you are and they like hearing from you.

“It’s not really surprising that, like search traffic, social media traffic tends to be very qualified,” says Maura Ginty, Senior Manager, Search and Social, Autodesk. “It can be small in volume, but it’s really qualified.”

Ginty’s team uncovered this insight by monitoring social media traffic to Autodesk’s website and analyzing the actions visitors took after arrival. Obtaining data around social media is not difficult, Ginty says. The hard part is using it.

“I think people end up feeling like the data is going to answer their question, but it’s the interpretation of all that data and the filtering of all that volume that really helps provide insight into what to do next,” she says. (Keep an eye on our Great Minds newsletter for an up-coming article on how to improve social media measurement).

Ginty’s team started synthesizing data to uncover the social impact of online marketing campaigns, in part, by using a tool created with Covario. For example, the team can calculate the velocity of a marketing message — the number of people a message reaches in a certain amount of time in social media — and combine it with a sentiment analysis. This information helps the team gauge how quickly messages spread, how people respond, and which efforts have strong social appeal.

“We’ve seen from different areas that a lot of the push of information will end up happening in the first 24 hours,” Ginty says.

Social media is a new channel with unique brand/customer interactions that can be tested and measured. I am excited to see how other industry leaders will start measuring and tweaking their social efforts to improve everything from brand image to conversion rates.

Are you doing any testing in social? Let us know in the comments…

Tracking, integrating marketing channels and smaller budgets are among CMO’s top challenges, according to a recent survey of more than 100 consumer-targeting CMOs. Aprimo and the Argyle Executive Forum conducted the survey on April 29, 2010.

Here are some highlights from the report:

1. More tracking is needed

39% of the CMOs said correlating marketing activities to revenues is the “most broken” area of marketing. 27% reported that the growing requirement for ROI and accountability is driving the most change in their marketing strategies.

Two likely contributors to this situation:
o The recent economy’s pressure on marketers to justify their budgets
o The unprecedented tracking potential offered by digital marketing

2. Multichannel marketing is challenging

Also related to tracking, 37% of the CMOs said their biggest challenge is integrating and tracking multiple channels. 27% said lack of marketing channel integration was the “most broken” area in marketing.

Integrating marketing channels and tracking customer interaction on an individual level can provide tremendous insight — but it’s difficult to achieve. Many marketing systems were not designed to play nicely together.

3. Budget woes continue

28% of the CMOS said “doing more with less” is their biggest challenge today, showing that the rising economy has not yet lifted marketing teams’ budgets and staff numbers to their previous levels.

What do you think of these stats? Are you experiencing something similar? Are you on a totally different page? Let us know…

I spoke with several paid search experts over the last two weeks for an article about timing PPC ads to optimize performance, and an interesting side-topic came up.

Seeing which PPC clicks are helping your bottom line is not always crystal clear. For example, a consumer may click an ad on Saturday and purchase the advertised item on Tuesday. These slow-converting, or latent clicks help drive sales. But by how much?

One way you can help figure this out is by looking to see whether an ad’s search phrase contains branded terms. Branded searches are likely driven by another marketing channel — because the consumer knew your brand name. Conversions on generic, non-branded search terms signal that your PPC ad had a much stronger influence on the sale.

You can track these slow-converting clicks using cookies — but even that can be challenging. Consumers often search the Web at work on one computer, and surf at home on another. Unless you’re able to connect those two machines, you’ll likely be missing some clicks that later become sales.

The lesson here is you should track the behavior of consumers who click your ads as well as you can. Doing so will give you a better idea of which clicks are driving delayed sales, and that information can help you better allocate your spending.

Have you found a good way to uncover slow-converting clicks? Has it helped you much? Let us know in the comments…

Measuring your marketing is the only way to know which efforts are working and which are wasting money. Even if you can’t measure every impact, you should track as much as possible.

After looking at some data from MarketingSherpa’s 2010 Social Media Marketing Benchmark Report, I wonder how many campaigns are only half-measured, with half their impact open to anyone’s guess.

When asked ‘What is your organization monitoring and measuring to quantify social media impact?’ 50% or more respondents said they were tracking:
o Visitors and traffic sources
o Followers, fans and members numbers
o Commentary about brand or product
o Sentiment around brand or product.

Fewer than 50% of respondents said they were tracking:
o Search engine rank
o Lead generation
o Progress toward social media objectives
o Engagement with influential bloggers, journalists, Twitterers, etc.
o Sales conversion and other ROI metrics
o Competitive share of social media coverage
o Criteria to identify and profile audiences

Astoundingly, only 35% of respondents said they were tracking sales conversion and other ROI metrics related to social media.

Getting more website traffic, Facebook fans and comments is very good. But if you’re not sure whether that’s having an effect on lead generation or sales, many executives will ask: what’s the point?

Marketers across the globe are finding use with social media. But if you want the rest of your organization to take it seriously and to invest more in the channel, you should learn as much about its impact as possible. The data talks.

If social media is helping you learn more about your audience, get data on how that knowledge is improving your marketing. If it’s helping your brand’s image, find a way to quantify it. Hypothetical evidence is as solid as a wet paper towel compared to hard data.

Is your team measuring its social media impact? If not, what’s holding you back? Let us know in the comments…