Top survey design pitfalls and how to avoid them

A lot of us students, marketers and business people are at some point faced with the task to create a questionnaire. After doing a few myself and helping colleagues and friends in this process, I’ve noticed how hard it is to design usable and purposeful surveys. Whether it is for industry-driven market research or academic studies, I think some basic principles should never be forgotten.

Here is my list of top survey mistakes and how to remedy then.

1. Cramming it all in

Based on the principle of ‘you’re never too careful’, survey designers often think: “Just in case it might be useful, I’m going to add this question in”. Make sure that every question is absolutely vital and that you are going to use it in your analysis. Let’s be honest, questionnaires are often a pain for respondents. If a question is there “just in case”, remove it. Be concise and purposeful.

2. Forgetting the research question

We often get sidetracked when writing a questionnaire. Always having in mind the research objectives we aim to address is key (assuming you have clear research objectives). The questionnaire aims to “measure” the concepts you’re interested in. Not more, not less. Always match a questionnaire item with the objectives of the study. If there is no match, the question should either be re-formulated, or deleted.

3. Forgetting the respondent

The questions you have in your mind as an investigator don’t necessarily make sense to the respondent. Think of the language they use, the words they are familiar with, their personal and social context; their education level. Avoid complex questions.

4. Starting with specifics

Questionnaire should go from general to specific questions, from simple to more complex. For instance, if you aim to test the interest for a product, first explore general interest, then see if the needs it aims to tackle are present, go on to ask about the specific attributes and functionalities of the product, maybe then its price too (always keep sensitive questions at the end not to scare respondents away).

5. Not knowing how you are going to analyse the results

The way you design a questionnaire influences the type of analysis you make and insight you get from it. A multiple-choice question is not analysed in the same way as a Likert scale, or as a semantic differential question. Did I lose you here? It means that you need to think of your question type, scale type and answer strategy carefully to know how to analyse the data. It might seem daunting but have a look at the books I recommend below.

Following these tips will help you enhance your survey design skills. There is much more to surveys than this, and I am still learning too, but this should be a good start!

For more details, see the list of books below.

Additional readings about survey design:

• About questionnaire design only: Ian Brace, 2013 – really comprehensive, addressing all survey-design issues.
• About marketing research: Hair, Bush and Ortinau, 2009 – on survey design in the broader marketing research agenda and with a focus on digital environments.
• About general business research: Bryman and Bell, 2015 – on survey design within general business research. Top reading for business students and practitioners.
• About web surveys: Don Dillman, 2011 – specifically addressing web surveys.