MarketingSherpa Blog

As marketers, we see lots of benchmark data and statistics that we base our business decisions on.

At MarketingSherpa, we recently conducted a nine-month study on the state of ecommerce.

You’ll see the results of our research conducted with 4,346 marketers across 95 in-depth charts.

Obviously, this data didn’t come out of thin air. There was a survey that our MECLABS research team carefully constructed to gather those insights.

Crafting effective surveys is potentially the most important part of collecting useful data, whether you’re fielding research for a report or simply gaining customer feedback.

Diana Sindicich, Senior Manager, Data Sciences, MECLABS (parent company of MarketingSherpa), played an integral part in the MarketingSherpa Ecommerce Benchmark Study and provided some tips on how to produce the most effective survey for your needs.

Survey Tip #1. Evaluate your situation

There’s a good time, and a not-so-good time, for everything. This rule of life applies to surveys as well.

In surveys, situations may exist for you that make it a good idea to field a survey, Diana explained.

This could include scenarios of when you want to understand your customers’ motivations or characteristics. Maybe you’re looking to expand your product lines and want to know what your customers would like to see offered.

On the other hand, there are times when a survey may not be the best idea for what you want to accomplish. Perhaps you have a very personalized service with a small group of customers. Surveys can be perceived as impersonal — conversely, an interview would make the customer feel special and valued.

Survey Tip #2. Understand bias in surveys

As much you try to make your survey as scientifically accurate and objective as possible, without random sampling, it’s not accurate.

Random sampling, by its definition, is a subset of individuals chosen from a larger set, and each individual is chosen randomly and entirely by chance.

No survey is truly random, and that is even truer in marketing. You’re probably surveying your customers, or like the MarketingSherpa Benchmark Studies, your audience.

As Diana explained, you cannot force people to take your survey, and you’ll only get responses from people who want to participate.

“The audience that you get is dependent on the incentive you offer, be it perceived or monetary value,” Diana said.

Perceived value, in the example of the Benchmark Study, would be leveraging the fact that respondents would be contributing to research. These incentives will affect the types of people you get to take your survey.

Always understand the predispositions your survey respondents most likely have. The tendency is to see some responses at the extreme ends of the spectrum — either very positive or very negative:

• The happiest people might respond
• The unhappiest will respond
• The neutral people won’t bother without perceiving value to participate in the survey

Understand that survey results are a measure of the respondents’ perceptions of your topic.

Design and analyze the survey with that in mind. If you can identify extreme groups, such as happiest and unhappiest customers, you can analyze the responses to other questions separately, often leading to very helpful insights into why customers are behaving as they do.

Survey Tip #3. Conduct background research and formulate research questions

Before you start to build your survey, utilize what you already know that could help your research, and formulate your research questions.

Finding information from past surveys, online review sites and third-party information on industry trends or population characteristics could hold troves of valuable information, Diana advised.

Based on what your background research reveals, what do you want to find out? What do you plan to do with the information you gain through the survey?

This is when you should develop your research questions. These are different that your actual survey questions, which we’ll address in the Tip #5. A research question should be something you want to answer through your survey. This might include questions such as:

• What confuses visitors on this page?
• What kind of people are going to this site?
• What information are prospects using in their decision process?

Survey Tip #4. Finalize methodology

Two of the biggest questions in survey methodology include: Where will your responses be solicited? And, how?

According to Diana, depending on the research questions you developed, it might become obvious to you at this point who can tell you what you need to know.

As for number of responses, “ideally, you want a lot, but a lot is relative,” Diana explained.

For surveys with primarily multiple choice questions, Diana advised, “You want to have a big enough number of responses that even the most unpopular answer options receive some responses.”

It is not unusual to see response rates on emailed surveys below 10%, so start with as big a list as possible.

Survey Tip #5. Construct your survey questions

Diana suggested for every research question you have, write down all of the possible survey questions you can think of. Don’t worry about having different versions of the same question; this is all part of the process of finding the best way to ask a question.

As you’re crafting your questions, remember to not lead your respondents to answer one way or another.

Diana gave a great example of taking a leading question and making it more objective:

Question: In which stores do you shop for shoes (e.g., Rack Room)?

This question would prompt many people to respond they shop at Rack Room because you’ve just reminded them about that store.

To change this question to make it non-leading, she suggested asking instead:

Where do you shop for shoes?

This question can be asked without providing a set response or giving a long list of store names that are in randomized order for each respondent.

This way, the respondent is recalling their experience rather than being led to respond a certain way.

Don’t forget to include “Other” and “N/A” as a response, as some survey takers may simply not know the answer or do not fit into the ones you have given. If they don’t see the answer that fits them and have no alternative response choices, they could leave the question blank or pick something randomly that won’t be accurate.

Survey Tip #6. Practice fielding your survey

Test your survey before you begin giving it to your intended audience. By doing a trial run, you can see how the survey will be experienced by people who have never seen it before.

Diana advised finding a group between five and 10 people to take your survey, so you can discover things like:

• How long did it take to complete the survey?
• Were any questions confusing?
• Were there any technical glitches?
• Did all of the questions record properly when data was exported?

You might also like

Infographic: Customer experience in the digital age [More from the blogs]

Email Marketing Research: What information will help you do your job better? [More from the blogs]

2 Vital Questions Every Marketer Should Ask of Lead Gen Forms [More from the blogs]

Small-Business Marketing: 5-step email survey process leads to 600% revenue growth [Case study]

Categories: Marketing Tags: benchmark report, customer experience, customer feedback, data, survey