TIPS FOR DESIGNING A SURVEY QUESTIONNAIRE
by Richard A. Krueger
Here's a collection of tips from survey experts. Consider these when developing your y questionnaire.
- Use an up-to-date mailing list
Seek a list that is trusted by experts, up-dated regularly and complete
- Use a personalized cover letter
The one-page cover letter on letterhead stationary should:
Present the questionnaire in an appealing format
- Explain what the study is about and its social usefulness
- Explain why the respondent is important to the study
- Promise confidentiality and explain of the identification number
- Include phone number to call if questions arise
- Thank respondent for participating
- Be signed by sender with blue ball point pen
Use quality questions
- Format should give a professional appearance and convey the impression that it is well thought out and important
- Use fonts that are easy to read
- Avoid a cluttered appearance -- Leave open spaces
- Establish a consistent vertical flow for questions
- Provide directions for how to answer. Brief directions can be placed at the beginning of the survey and more specific instructions can occur before categories of questions or specific questions
Use appropriate response categories for your questions
- Clearly stated - Avoid jargon
- Short but not cryptic or vague -- Avoid long sentences
- Unidimensional (not double-barreled, e.g. "Was the service fast and friendly?")
- Reasonable to answer (respondent knows the answer)
- Limit the "skip" questions (If yes, go to question xx)
- Opening questions should be easy and non-threatening
- Demographic questions such as age, marital status, and income are usually considered personal and should be avoided or placed at the end of the questionnaire. Don't ask them if you don't know how you will use the data
- Place questions in a logical order. Group questions of a certain category together and use headings as appropriate
- Some questions may seem similar. If this occurs underline the key word so the respondent can easily spot the differences. (For example: "helpful" and "useful" are similar but different)
Your response categories should be:
Consider your analysis strategy before you finalize the survey format
- Non-overlapping (each category is distinct from the others)
- Exhaustive (there are no gaps in the range of response choices)
- Appropriate to the question (the response choice is logical given the question)
- Consistent (all response choices are consistent with each other)
- Favor rating over ranking (Ranking means they place items in sequence such as first choice, second choice, etc. Ranking cannot be averaged. Rating means that they give an answer using a predetermined scale such as: Excellent, Good, Fair, Poor. Choices that are rated can be averaged.)
- Provide directions for how to answer. Don't assume that the respondent will automatically know how to answer. An example may be helpful. Are they supposed to check, circle, or darken in the box?
- When selecting response format remember that parenthesis ( ), braces [ ], and circles O, are preferred to horizontal lines ______.
Pilot test the instrument
- Make revisions to speed up computer data entry. Number the questions
- Favor "rating" questions to "ranking" questions. Rating responses are usually treated as interval data whereas ranking responses are considered ordinal
- Decide on analysis strategies for open-ended responses
Aim for a high response rate
- Does each question measure what it is supposed to measure?
- Are all the words understood?
- Do all respondents interpret the question in the same way?
- Are all response choices appropriate?
- Do respondents correctly follow directions?
- Is the range of response choices actually used?
- Does the mail-out questionnaire create a positive impression that motivates people to respond?
Consider a strategy for non-respondent follow-up
- A small sample with a high response rate is preferable to a large sample with a low response rate. Here is a helpful website with suggestions on sample size: http://agpublications.tamu.edu/pubs/ee/d1340.pdf
- Use the "Babbie" response rate rule of 50-60-70. 50% response is "adequate", 60% is "good", and 70% is "very good"
- Set a due date 7 to 10 days after receipt of survey
- Include an identification number on each survey
- Include a stamped addressed return envelope
- Send a post card thank you \ reminder which arrives on due date
- Send a replacement survey to non-respondents one week after due date
- Telephone a sample of non-respondents and compare their responses to those obtained in the mail-out process
If you would like to read more about surveys, consider these references:
Babbie, Earl R. (1979) The Practice of Social Research. Belmont, CA: Wadsworth.
Dillman, Don A. (1999) Mail and Internet Surveys. New York: Wiley.
Sudman, Seymour and Norman M. Bradburn. (1982) Asking Questions: A Practical Guide to Questionnaire Design. San Francisco: Jossey-Bass.
University of Kentucky Extension website: Documenting Practice Change with Mailed Questionnaires. http://www.ca.uky.edu/agpsd/soregion.htm
Back to EVALUATION