How NOT to Complete a Survey

Based on our office's work in analyzing perception data from surveys, we notice a lot of things that people do when filling out a survey that make the analysis difficult, or worse, make responses void. Here is our short list of reminders of how to complete a survey if you want your responses to be included in the analysis.

1) When the question asks you to circle or check one response only - check or circle only one response.

Often times, an organization wants to know your top choice or the best _____, so you are asked to choose one only. When you check more than one because you think there are 2 best or 3 favorites, then the item is unusable because the researchers cannot pick which one was the "best" of those selected.

In items than have a scale (e.g., a 6-pt agreement - strongly agree to strongly disagree), we tend to see responses where folks circle more than one (a big circle around 2 and 3). Sometimes we even see an "X" in between two numbers. These are no-nos, because there is no option of 2.5 (we'll cover ordinal vs. interval scales in a later blog), and we don't know if the respondent was really more a 2 or more a 3. So please choose one, otherwise your response is unusable.

2) If the questions asks you to rank the top 3 choices, they need to be ranked - and only up to 3.

Items that ask you to rank are different from items asking you to check all that apply. If you are asked to rank the top three (1, 2, and 3), and you just check off three options, then there is no way to know which was first, second, or third choice, rendering the response void. In one recent survey, we had to exclude 25% of responses to a ranked item because folks just checked three instead of ranking them as the question prompt asked.

The same rule applies if you rank items: 1, 2, 2 (suggesting the second and third are tied). We can use the 1, but we don't know which one of the others is really second.

3) When there is an ID code asked, please fill it in completely.

To ensure confidentiality and to attempt to allow for anonymity, we often ask for ID code (a combo of name/phone number). These codes are important for pre-post studies so that responses can be matched. If the codes are not filled out completely as per the instructions in the pre- or post- setting, the surveys cannot be matched and both the pre- and post- survey may have to be excluded from the analysis.

4) If you have comments, place them in the additional comments section or at the end of the survey.

When there are tiny comments crunched into a scale survey item line or squeezed between two categories, 1) it is difficult to read and 2) there isn't a location in the database for these comments so they may not be considered. To make sure your comment is clear and counted, it is best to place them at the end of the survey.

5) Please print!

Chances are that if a respondent took the time to respond to open ended (fill in) items, the comments are important for the analyst to include and for the organization to hear. For this reason, we ask that folks type or print their comments. There have been many comments that were illegible due to the cursive or shorthand that respondents used. This is even more critical if you are scanning or faxing a document.

*****

We hope that if you see yourself in any of these comments that you are not offended and now realize why survey responses are asked for in particular ways. If you have a question about how to complete the survey, there always should be a contact person listed on the survey or on the survey cover letter (if applicable). I know our office would prefer phone calls to ensure valid responses instead of having to code responses as "missing data."