Program Evaluation in Practice

Earlier this month we described the nine-step program evaluation process, a measurement and evaluation best practice that organizational development firm Brighter Strategies has helped many nonprofit agencies to implement. To give you a real-world idea of what this process looks like in action, below we revisit Sue Ito and her Young Adult Education Program at Jefferson County Library.

Fast-forward a few years since we last heard from Sue. Now the YAEP is a well-established and thriving program at JCL. However, the library itself has experienced recent change. County government funding cuts have affected many community organizations, and JCL is no exception. The YAEP is at risk of losing much of the technology support it has relied on for success.

Program evaluation for change

As we’ve learned, program evaluation is the process of collecting information about a program or some aspect of a program to make necessary decisions about that program. Sue believes a process-based program evaluation, which determines how a program operates and achieves its results and is beneficial when long-standing programs experience recent change, will help YAEP to survive and even thrive through budget constraints.

Sue mobilizes her program planning team, and they outline their program evaluation strategy going forward. They revisit the logic model they created to ensure they have a clear picture of the original program, including inputs, outputs, and outcomes. They review trending outcome data since the program’s inception, to determine if any results have slipped. The program “picture” is relatively unchanged, and the program outcomes have overperformed against their goals each year.

Now, with the external environment changing a handful of the program inputs that rely on monetary resources, Sue asks the team to remove these items from the logic model. This brings total inputs from five to a mere two: volunteer staff and library space.

Inputs
What we invest

Volunteer staff

 

Paid staff

 

Computers

 

Career development materials

 

Library space

With this bleak backdrop, the team creates a data collection plan to determine if the YAEP could achieve its existing performance outcomes with barebone resources. Through one-on-one interviews with program participants, volunteers, JCL employees, and community representatives, Sue’s staff learns that collective passion for the YAEP is at an all-time high. The program has helped the community to strengthen relationships with its at-risk young adult population, and many volunteers have made personal connections with participants as a result.

Sue’s team takes its data digging one step further. It gathers a focus group with a variety of stakeholders to ask the question: How could Jefferson County Library’s Young Adult Program survive if the community had no money to invest in it? This time-efficient process produces a variety of feedback, which Sue’s team incorporates within its revised logic model. They can now show how YAEP will be able to create the same outputs and subsequent outcomes with zero funding from the library. Volunteers are engaged in the program to such an extent that they are more than willing to contribute their time, tools, and creativity to keep YAEP alive.

Inputs


What we invest

Outputs


What we produce

Short-term Outcomes

Short-term impact

Mid-term Outcomes


Mid-term impact

Long-term Outcomes


Long-term impact

Volunteer staff

 

Library space

 

Volunteer staff-owned technology, including computers and mobile devices

 

Career development materials created by volunteer staff

 

Mentoring relationships between volunteers and program participants

Career education and resources for disadvantaged youth

 

Technology training for program participants

 

Job skills training for program participants

 

Life skills coaching

 

Safe space for disadvantaged youth to meet weekly

New learning, such as:

 

IT knowledge gained

 

Job skills learned

 

Positive attitudes toward higher education developed

New actions, such as:

 

Participants apply to post-high school colleges or apprenticeship programs

 

YAEP participation grows

 

JCL receives increased community funding

 

 

New conditions, such as:

 

Rate of college-educated youth in community increases

 

Young adult crime rate decreases

 

Community reputation is elevated

Sue is thrilled. Her team’s investment in program evaluation has helped to redirect resources for maximum impact during change at JCL. She completes the process by creating an action plan for next steps, including new volunteer recruitment and training. She shares evaluation results with all individuals involved in the process, as well as vested community stakeholders.

For more details on program evaluation, check out organizational development consulting firm Brighter Strategies’s resource guide, “Evaluating Performance Outcomes: A Guide to Implementing Program Evaluations.”

Share:

Leave a Reply

Your email address will not be published. Required fields are marked *