The Role of Outcome Measurement, Part I

Earlier this month we outlined a side-by-side comparison of outcome measurement and program evaluation. In the next two blog articles, we unpack the outcome measurement process specifically.

Outcome measurement: Steps One Through Five

  1. Assess your organization’s readiness.

    Outcome measurement is the determination and evaluation of a program’s results, and their comparison with the intended or projected results. This definition requires a starting point. Check out Brighter Strategies’s evaluation products here for help assessing your agency’s “Point A.”

  2. Select the program and create a plan.

    After understanding the state of your organization, decide which program you will measure. Choose a program that is undergoing a certain change or requires transformation. Outline a specific plan for outcome measurement, including a timeline with milestones by which it will be completed.

  3. Describe the program’s mission and activities using a logic model.

    Visually depict the program you choose with a logic model, such as this:

Copyright Isaac Castillo, 2104.

Complete the logic model with your outcome measurement team during a series of working sessions.

  1. Identify the program’s intended results, or outcomes.

    Program outcomes comprise the second half of the logic model. These goals are the “meat” of outcome measurent, and the focus of subsequent steps of the process.

  2. Identify indicators of success for each outcome.

    Bring your program’s outcomes to life by assigning each goal a metric, which ensures completion can be successfully measured.

Outcome measurement: The Story of Sue Ito

To illustrate this first half of the outcome measurement process, we are introducing Sue Ito, Director of Planning and Performance at the Jefferson County Library (JCL). Sue has been tasked by the County Board of Directors to report a set of agency outcomes on an annual basis.

To begin, Sue uses a “Start With the End in Mind” planning product, which shows that JCL’s Young Adult Education Program (YAEP) is ripe with opportunity, and much of the organization is eager to support its growth. Sue decides to conduct outcome measurement for YAEP and forms a cross-department team to assist her.

The team meets weekly and for the first month focuses its working sessions on creating a logic model to better visualize and understand the current program. Below is a snapshot of the YAEP inputs, outputs, and outcomes.

Inputs

What we invest

Outputs

What we produce

Short-term Outcomes

Short-term impact

Mid-term Outcomes

Mid-term impact

Long-term Outcomes

Long-term impact

Volunteer staff

 

Paid staff

 

Computers

 

Career development materials

 

Library space

Career education and resources for disadvantaged youth

 

Technology training for program participants

 

Job skills training for program participants

 

Life skills coaching

 

Safe space for disadvantaged youth to meet weekly

New learning, such as:

 

IT knowledge gained

 

Job skills learned

 

Positive attitudes toward higher education developed

New actions, such as:

 

Participants apply to post-high school colleges or apprenticeship programs

 

YAEP participation grows

 

JCL receives increased community funding

 

 

New conditions, such as:

 

Rate of college-educated youth in community increases

 

Young adult crime rate decreases

 

Community reputation is elevated

 

Sue reviews the logic model with her team and realizes the outcomes are strong, but vague. How will the new learnings, actions, and conditions be proven? The team creates specific metrics, or indicators of success for each, captured below.

Short-term Outcomes

Short-term impact

Indicators

Mid-term Outcomes

Mid-term impact

Indicators

Long-term Outcomes

Long-term impact

Indicators

IT knowledge gained

 

60 percent of participants show evidence of increased IT knowledge on pre- and post- program surveyIncrease in college or apprenticeship program participation

 

80 percent of participants apply to higher education programsRate of college-educated youth in community increases

 

 

Number of college-educated youth in community increases by 20 percent
Job skills learned

 

100 percent of participants report acquisition of new job skillsIncrease in YAEP participation

 

Program enrollment rate grows by 30 percent in the first yearYoung adult crime rate decreases

 

 

Young adult crime rate decreases

by 5 percent

Positive attitudes toward higher education developed75 percent of participants cite intentions to pursue higher education opportunitiesIncrease in community funding for JCL

 

JCL receives a $25,000 grant from the local government to support growing YAEPCommunity reputation elevatedAverage real estate prices in Jefferson County grow by $10,000 year-over-year

What are your thoughts about the above outcomes and indicators? Which are easy wins, and which require a great deal of work to achieve? Please share your feedback in the comment section below.

In our next blog article, we’ll wrap up the outcome measurement process with our story of Sue and YAEP. Stay tuned!

Share:

Leave a Reply

Your email address will not be published. Required fields are marked *