Summative Evaluation

This is Section 5 of the Evaluation Toolkit
for the TA & D Network.
Read about the toolkit.

Summative evaluation can help you determine outcomes and longer-term benefits of your dissemination efforts. Outcomes generally have to do with target audience reports of how they benefited and used what you disseminated to them.

What do you want to learn?

What do you want to learn about your dissemination work? Summative evaluation can help you. Here are a few examples to illustrate. You might want to know:

How teachers benefited and used what they learned from an on-line module about differentiated instruction featured on your project’s website.

How parents used one of your project’s information tools about the family’s roles and rights in the IEP process to get better services for their child.

If there was a strengthening of partnerships and collaboration with other TA & D projects in conducting joint dissemination activities.

You could also look at the extent to which direct recipients of your project’s dissemination work engaged in their own dissemination (i.e., to others) with your materials. This could be a good outcome indicator that dissemination efforts are having a broader effect, beyond your project’s more immediate and direct audiences.

Rigor in summative evaluation

Summative evaluation usually focuses more on using quantitative measures rather than qualitative measures, which facilitate collecting more precise information about people’s behaviors and actions they took.

Summative evaluation can be a complicated and expensive undertaking, especially if a high degree of research rigor is desired. Rigorous summative evaluations seek to establish impacts of programs or interventions – i.e., that the program or intervention was the direct cause of impacts. Demonstrating these types of impacts generally requires sophisticated research methods including random assignment of those receiving the program to those that did not, comparison groups, large numbers of individuals to participate in the research, and high-level statistical analyses.

Most TA & D projects do not have the financial or personnel resources to plan and conduct rigorous impact type studies. However, there are some less rigorous evaluation approaches which can yield potentially useful information to TA & D projects about the possible benefits and outcomes of their dissemination work.

Looking at benefits and outcomes of dissemination

Outcomes are typically categorized as short-term, intermediate, and long-term.

Short-term outcomes are concerned with the immediate benefits and effects of your dissemination work, such as changes in knowledge, awareness, beliefs, and attitudes.

Intermediate outcomes are more concerned with benefits and effects that usually take longer to occur and are often related to behavior change or specific actions that a person takes.

Long-term outcomes are generally about the broader and sustained effects and benefits, and follow logically from the shortterm and intermediate outcomes.

TA & D projects may want to focus summative evaluation work on the short-term and intermediate outcomes, especially if resources for evaluation are limited.

Examples | Here are some examples of short-, intermediate-, and long-term goals for two different user groups.

Special education teacher examples

Short-term goal: First year special education teachers increase their knowledge, awareness and skills about the instructional needs of students with learning disabilities, from an on-line module developed by your project.

Intermediate goal: First year special education teachers apply the knowledge and skills they acquired, from your on-line module, by providing high quality instruction to their current class of students with LD.

Long-term goal: Special education teachers demonstrate ongoing capacity and increasing skill development over successive school years to deliver effective instruction to students with LD.

Parent examples

Short-term goal: Parents of children with autism gain knowledge from your project website resources about parenting techniques and skills to use with their child.

Intermediate goal: Parents apply the knowledge and skills consistently and effectively to improve child behavior and family relationships.

Long-term goal: Parenting skills and improvements in family relationships contribute to greater school success for their child.

Suggested citation

This webpage is an excerpt from the evaluation toolkit produced by NICHCY. The suggested citation is:

To the entire toolkit
Sawyer, R. (2012). Toolkit for the OSEP TA & D network on how to evaluate dissemination: A component of the dissemination initiative. Washington, DC: National Dissemination Center for Children with Disabilities.

To this section/webpage on summative evaluation
Sawyer, R. (2012). Summative evaluation. In Toolkit for the OSEP TA & D network on how to evaluate dissemination: A component of the dissemination initiative (pp. 9-10). Washington, DC: National Dissemination Center for Children with Disabilities.