For the past couple of months I’ve written about program evaluation—the three types, how to create a logic model, and the importance of designing clear outcomes and indicators. Today I’ve saved the best news for last: Rest assured that program evaluation is not a one-time event. It is neither prescriptive nor linear, so you get more than one shot at it.
Program evaluation is a cyclical process that follows specific steps and supports a continuous quality improvement approach. Each program will enter the evaluation cycle at a different spot. For example, your evaluation may begin with data your organization has already gathered or with the creation of a logic model. This starting point depends on where you are with your program and when you decided to engage in the evaluation process.
The Nine-Step Cycle
At Brighter Strategies, we have adopted a nine-step cycle that makes program evaluation simpler for nonprofit leaders to implement. Read on to learn how to apply this cycle in your agency.
- Assess your readiness: How prepared is your organization for an evaluation? Do you have a culture of data-driven decision making? If not, it’s difficult to justify an investment in evaluation to your senior leaders or effectively use results to inform program changes. If your organization lacks a data-driven culture, you may not be ready for evaluation yet. In this case, work to create an organizational culture that supports evaluation before diving into the cycle.
- Start with the end in mind: Identify clear program outcomes. Too often organizations think about programs they want to implement, that their donors may like, or that “feel good.” But it’s important to prioritize outcomes before activities. Think about the results you want to create and identify the best programs to achieve those outcomes or the connections between current programs and their desired outcomes. If such connections do not exist, it may be time to modify existing programs.
- Develop a data collection plan. Choose data gathering methods that support your organization’s needs and the specific outcomes you have identified in step two. Gain buy-in for this process by inviting stakeholders to participle, explaining how you will use the data, and sharing your findings. Finally, use data-gathering channels that are formal (for example, peer-reviewed journals) and informal (such as participant feedback following a conference session).
- Identify resources. Identify the relationship between the outcomes you wish to accomplish, and the resources required to achieve them. Determine if your organization currently has access to the appropriate resources and can support the program activities you plan to implement. If resources are lacking, choose which activities to forego or what new resources to acquire to drive desired results.
- Determine program outcomes. Program outcomes are often confused with outputs, but the two are distinctly different. Outcomes indicate program effectiveness; change in knowledge, skills, attitude, or behavior; stakeholders’ experiences; and the benefits of the program. Outputs indicate program efficiency; units of service produced; and the tangible value resulting from the program. Learn more about writing outcomes here.
- Review and analyze data. Focus on using the data review to answer your evaluation questions. When analyzing qualitative data (observable data that deal with descriptions), group comments into categories and themes, highlighting program strengths and areas for improvement. When analyzing quantitative data (measurable data that deal with numbers and definitions), perform various statistical calculations like ranking, mean, median, and mode. Compare data to establish targets for each indicator; determine progress against local, state, national, and other program benchmarks; and track trends among reporting periods. Most importantly, draw conclusions by looking at multiple data sets rather than focusing too heavily on a single data point.
- Create a logic model. A logic model is a systematic and visual way of presenting relationships between the need for a program’s existence, resources allocated to meet the need, day-to-day activities that make up the work of the program, outputs, and outcomes. A logic model implies an “if-then relationship” between each of these areas. Learn more about creating a logic model here.
- Track and use outcomes and evaluation results. The bigger picture impact of program evaluation is its ability to inform your organization’s strategic management approach. This is a critical step that helps to justify program resources. Form small working groups to track outcomes and examine reasons for unusual results over time.
- Integrate results. Use results for strategic planning to ensure your program works to its fullest potential and hits intended targets. For example, brainstorm possible program modifications to help better achieve outcomes or determine how effectively any modifications have improved services. Finally, communicate results by disseminating the evaluation report to relevant internal and external stakeholders.
We offer a free e-book with practical resources to help you implement this nine-step cycle in your organization. For more information about program evaluation, check out my blog post, “Program Evaluation and Your Organization.”
If you are ready to get started implementing the program evaluation cycle in your nonprofit and could use some guidance and support, contact us today.