Notes on Questionnaire Design

Questionnaires should be able to collect information that is:

• Valid:  measures the quantity or concept that is supposed to be measured
• Reliable:  measures the quantity or concept in a consistent or reproducible manner
• Unbiased:  measures the quantity or concept in a way that does not systematically under- or overestimate the true value
• Discriminating:  can distinguish adequately between respondents for whom the underlying level of the quantity or concept is different

A Guide to Making Your Questionnaires

1. Write a detailed list of the information to be collected and the concepts to be measured in the study.     Are you trying to identify:

• Attitudes
• Needs
• Behavior
• Demographics
• Some combination of these concepts

2.Translate these concepts into variables that can be measured.

• Define the role of each variable in the statistical analysis:
• Predictor - independent or experimental variable
• Confounder - other variables that have an effect on the dependent variable that the researcher did not account for 
• Outcome - dependent variable

3. Review current literature to identify related surveys and data collection instruments that have measured concepts similar to those related to your study’s aims. Saves development time and allows for comparison with other studies if used appropriately.

4. Determine the mode of survey administration:  face-to-face interviews, telephone interviews, self-completed questionnaires, computer-assisted approaches.

5. Format the draft as if it were the final version with appropriate white space to get an accurate estimate as to its length – longer questionnaires reduce the response rate.

6. Place the most important items in the first half of the questionnaire to increase response on the important measures even in partially completed surveys.

7. Make sure that a question provides the necessary information.  If a question does not address one of your aims/objectives/sub-problems, discard it.

8. Refine the questions included and their wording by testing them with a variety of respondents.

• Ensure the flow is natural.
• Verify that terms and concepts are familiar and easy to understand for your target audience.
• Keep recall to a minimum and focus on the recent past.

Guide to Improving Your Questions

Question:  How many cups of coffee or tea do you drink in a day?
Principle:  Ask for an answer in only one dimension.
Solution:  Separate the question into two –
(1) How many cups of coffee do you drink during a typical day?
(2) How many cups of tea do you drink during a typical day?

Question:  What brand of computer do you own?
(A) IBM PC
(B) Apple
Principle: Avoid hidden assumptions.  Make sure to accommodate all possible answers.
Solution:
(1) Make each response a separate dichotomous item
Do you own an IBM PC? (Circle:  Yes or No)
Do you own an Apple computer? (Circle:  Yes or No)
(2) Add necessary response categories and allow for multiple responses.
What brand of computer do you own?  (Circle all that apply)
Do not own computer
IBM PC
Apple
Other

Question:  Have you had pain in the last week?
[  ] Never [  ] Seldom     [  ] Often     [  ] Very often
Principle:  Make sure question and answer options match.
Solution:  Reword either question or answer to match.
How often have you had pain in the last week?
[  ] Never     [  ] Seldom     [  ] Often     [  ] Very Often

Question:  Where did you grow up?
Country
Farm
City
Principle:  Avoid questions having non-mutually exclusive answers.
Solution:  Design the question with mutually exclusive options.
Where did you grow up?
House in the country
Farm in the country
City

Question:  Are you against drug abuse? (Circle: Yes or No)
Principle:  Write questions that will produce variability in the responses.
Solution:  Eliminate the question.

Question:  Which one of the following do you think increases a person’s chance of having a heart attack the most?  (Check one.)
[  ] Smoking [  ] Being overweight [  ] Stress
Principle:  Encourage the respondent to consider each possible response to avoid the uncertainty of whether a missing item may represent either an answer that does not apply or an overlooked item.
Solution:  Which of the following increases the chance of having a heart attack?
Smoking: [  ] Yes   [  ] No   [  ] Don’t know
Being overweight: [  ] Yes   [  ] No   [  ] Don’t know
Stress: [  ] Yes   [  ] No   [  ] Don’t know

Question:
(1) Do you currently have a life insurance policy?  (Circle:  Yes or No)
If no, go to question 3.
(2) How much is your annual life insurance premium?
Principle:  Avoid branching as much as possible to avoid confusing respondents.
Solution:  If possible, write as one question.
How much did you spend last year for life insurance? (Write 0 if none).

Source:
Jenkins, Cathy. Questionnaire Design. Powerpoint Presentation.