Surveys

Section 6 continues the discussion of data
collection methods. This time—surveys.
Read about the toolkit.

Surveys are useful for collecting more quantitative data (e.g., numerical counts, frequencies, percentages). This webpage on surveys addresses:

When surveys are useful
How will you collect the data?
Establishing the baseline
Advantages
Disadvantages
Guidelines
Example of a target audience website survey

When surveys are useful

Surveys can be a good method for collecting data about needs, experiences with program implementation, as well as outcomes – including how target audiences have benefited and used information and resources disseminated to them. Surveys are also useful in assessing:

attitudes,
decisions that participants have made,
behaviors and practices,
lifestyle choices,
affiliations (e.g., with disability groups), and
demographics (e.g., specific disabilities).

Surveys are often used with larger number of participants than is the case with focus groups or individual interviews. For example, surveys can more easily be fielded to hundreds or even thousands of persons (via websites, email links to SurveyMonkey) in one or more of your target audience groups, in an effort to get larger and even representative samples – and at potentially lower costs than conducting numerous interviews or focus groups.

How will you collect the data?

Surveys can be conducted via regular mail, telephone, face-to-face, email, and the web.

While web-based surveys have become increasingly popular, they may result in lower response rates that face-to-face and telephone surveys. Nevertheless, web surveys can be less costly than other survey approaches, but are limited to those with internet access.

Establishing the baseline

Surveys are also a good evaluation method for establishing baselines against which future comparisons can be made and to analyze trends across time (e.g., compare the information needs of parent support groups from year to year; identify changes that may have occurred in the instructional practices of teachers who participated in a training).

Advantages of surveys

Can be fairly easy to administer – in person, on-line, via email.

May be less expensive and demanding on personnel resources than other methods such as interviews or focus groups.

Can be administered to large numbers of persons, and particularly useful in situations when persons are geographically dispersed.

A great deal of information can be gathered in a relatively short period of time, and survey data can be aggregated and analyzed by computer processing.

Can eliminate or greatly minimize the potential for sidetracking or collection of less relevant information than can occur in other, more interactive methods (e.g., interviews and focus groups).

May be viewed by respondents as a much more confidential data collection process (compared to interviews and focus groups), thus encouraging respondents to provide more accurate and honest responses.

Disadvantages of surveys

The survey development process can be complex and time consuming, especially if there is need to collect a substantial amount of information.

If a survey is very short, it may not elicit much useful information. If it is too lengthy, respondents may not take the time to complete the entire survey.

There may be a lack of assurance that the survey questions or items were accurately understood by participants; however, pilot testing can help to alleviate this potential problem.

Responses to open-ended questions may be difficult and time consuming to analyze and summarize.

Low participation rates can result in a non-representative response rate, thus compromising interpretation of findings and use of results.

Guidelines

Use structured questions, with fixed-choice response options, rather than unstructured or open-ended ones in order to obtain uniformity and consistency in responses; this will assist in the analysis of survey responses. Where appropriate, supplement structured questions with open-ended ones to obtain more “indepth” responses.

When developing survey questions and items, the following should be considered:

1. Clarity is essential; avoid ambiguous items and wording.

2. Shorter, more concise items are preferable to longer items as they are usually easier to understand.

3. Avoid “double-barreled” items, which require the participants to respond to two or more separate ideas with a single answer.

4. Avoid highly technical terms or jargon.

5. Avoid biased or leading questions.

Pilot test the survey, with a small sample of individuals, to eliminate ambiguous or biased items, improve sequencing of questions and items, improve format, and ensure that overall length is appropriate.

Include introductory language, either written or verbal, that tells participants the purpose of the survey and assures them that their responses are confidential.

Include a seslf-addressed, stamped return envelope with “regular” mail surveys to boost response rates.

Send reminders–-one or more follow-ups—to participants. Reminders are usually needed to increase response rates.

Provide an incentive (e.g., free materials, small amount of money) to participants, if possible, to boost return rates.

If possible, complement the survey method with a qualitative method (e.g., interviews, focus groups), to obtain more “in-depth” information on topics covered in the survey.

Abbreviated example of a target audience website survey

Editor’s note: You may find it easier to read a PDF version of this example. If so, we’re pleased to offer it as stand-alone pages and as part of the full evaluation toolkit (see pages 19-22). Here’s the example of a target audience website survey, taken from NICHCY’s own website:

Recently you visited our project website. Our project provides information and resources on disabilities and special education. We would like to ask you to spend about 5-10 minutes answering a few questions about your experience on our website. Your responses to this questionnaire are completely confidential and will not be disclosed to others. Data from individual responses will be combined with many others to produce reports.

This questionnaire data collection, supported by the Office of Special Education Programs, U.S. Department of Education, is intended to assist our project in obtaining information from which to improve services. If you have questions about this data collection, please contact [person’s name] at [phone number] or at [person’s email address].

1. What type of information were you looking for on our website, today or on any previous visits to the website? (Check all that apply)

Information about….

___ My own disability
___ A family member’s disability
___ The disability of a person who’s not a family member
___ Early intervention services
___ Special education
___ Adult services for persons with disabilities
___ Disability laws (e.g., Individuals with Disabilities Education Act)
___ Resources in my state
___ Cannot remember
___ Other; please specify ______

2. Please indicate your level of agreement with each of the following statements about the information you obtained from our website, today and from any previous visits to the website.

Editor’s note: The “level of agreement” choices are listed next to each item and offer respondents these options: Strongly Agree | Somewhat Agree | Somewhat Disagree | Strongly Disagree| Not applicable/Not sure

The information was/is…

___ easy to access

___ easy to understand

___ relevant to an issue, need or concern I have

___ useful and applicable within my professional and/or personal life

___ of high quality

___ informative about resources available to address my needs

___ informative about where to go for more or different types of information

3. Have you made use of the information or resources from our website (e.g., from past visits to the website)? (Check only one response)

___ Yes (go to Question 4 )

___ Not yet, but I still intend to do so (go to Question 6)

___ No, and I don’t intend to use the information (go to Question 7)

4. If yes, please indicate how you have used information or resources from our website. (Check all that apply)

___ I’ve used it in my personal life.

___ I’ve shared/used it with my family members

___ I’ve shared/used it with those who provide education and/or services to me or a family member

___ I’ve used it my professional work

___ I’ve shared it with professional colleagues

4a. Briefly describe how you used the information you found on our website.

__________________

5. Please indicate your level of agreement with each of the following statements. Please select “Not applicable” if an item does not apply to you (e.g., you are not a parent of a child with a disability).

Because of the information/resources I found on the website, I have….

___ Learned more about specific disabilities or disability topics.

___ (Parents) Been more able to help my child with schoolwork.

___ (Parents) Been more able to help my child improve his/her behavior.

___ (Parent) Been more able to work with staff at my child’s school.

___ (School staff) Provided better instruction or services to students.

___ (Administrators) Provided better programs for those with disabilities.

6. If you intend to use the information from our website at some time in the future, how do you think you’ll use it? (Check all that apply)

___ I’ll use it in my personal life.

___ I’ll share/use it with my family members

___ I’ll share/use it with those who provide education and/or services to me or a family member

___ I’ll use it in my professional work

___ I’ll share it with professional colleagues

___ Not sure

6a. Briefly describe how you intend to use the information you found on our website.

__________________

7. Please indicate your reason(s) for not intending to use the information. (Check all that apply)

___ I didn’t intend to find information for a specific use when I visited the website

___ I don’t know how to use the information I obtained

___ I am waiting to see if I need to use the information I obtained

___ I got more useful information from another source

___ Other (please specify) ____________

8. How likely are you to visit our website again? (Check only one response)

___ Very likely

___ Somewhat likely

___ Somewhat unlikely

___ Very unlikely

___ Not sure

9. How likely are you to use our website as your main stop for information on special education/disabilities? (Check only one response)

___ Very likely

___ Somewhat likely

___ Somewhat unlikely

___ Very unlikely

___ Not sure

10. Would you recommend our website to others who might benefit from information on special education/disabilities? (Check only one response)

___ Yes, definitely

___ Yes, probably

___ No, probably not

___ No, definitely not

11. What suggestions, if any, do you have regarding changes, enhancements or improvements to our website?

Comment:_____________________________

12. Which of the following best describe(s) you? (Check all that apply)

___ 1. Early intervention provider (Birth to 2 yrs)

___ 2. Preschool staff (3-5 yrs)

___ 3. K-12 general education teacher

___ 4. K-12 special education teacher

___ 5. Related services provider (e.g., OT, PT, Reading Specialist, Counselor)

___ 6. Education, other school personnel (e.g., Teachers Aide; paraprofessional)

___ 7. K-12 student

___ 8. School administrator

___ 9. State education agency staff

___ 10. Local education agency staff

___ 11. Federal agency staff

___ 12. Person with a disability

___ 13. Parent/family member

___ 14. Post-secondary education faculty

___ 15. College/university student

___ 16. Parent organization staff

___ 17. Organization, disability

___ 18. Other; please specify ________________________________

13. Would you like to receive our monthly eNewsletter, which contains up-to-date information on issues and resources related to special education and disabilities? If so, please provide your email address below.

Your email address: __________________

Thank you very much for responding to this questionnaire. Your feedback is very important to us.

Suggested citation

This webpage is an excerpt from the evaluation toolkit produced by NICHCY. The suggested citation is:

To the entire toolkit
Sawyer, R. (2012). Toolkit for the OSEP TA & D network on how to evaluate dissemination: A component of the dissemination initiative. Washington, DC: National Dissemination Center for Children with Disabilities.

To this section/webpage on conducting surveys as a data collection method
Sawyer, R. (2012). Surveys. In Toolkit for the OSEP TA & D network on how to evaluate dissemination: A component of the dissemination initiative (pp. 17-22). Washington, DC: National Dissemination Center for Children with Disabilities.