Building Better Metrics
Your organization likely struggles with obtaining reliable and readable metrics. We have been discussing Fram and Talley’s advice for using imperfect metrics published in the October issue of Nonprofit Quarterly http://nonprofitquarterly.org/management/20701-using-imperfect-metrics-well-tracking-progress-and-driving-change.html.
One of the first points they establish is to have agreement with your board and stakeholders regarding what metrics you will obtain and we will look today at how you might find them. In our blog today we will examine the technical challenge by looking at the five-step process each example in our blog series used to develop their own metrics. Fram and Talley suggest the following:
- Agree on relevant outcomes: Metrics should be used to reflect organizational outcomes or impact, not activities or efforts. For example, one general outcome could be “To enable the student to refine and evaluate his/her occupational goal.” Or it might be “Build community support for preservation initiatives.” Or “Be the catalyst for stronger community ties.” Outcomes should focus on a desired change in the nonprofit’s universe rather than a set of process activities.
- Agree on measurement approaches: There are many possibilities for measurement. These include personal interviews, mail questionnaires, sampling data in client records, Internet surveys, comparisons with other agencies, peer or outside consultant visits, and comparing the organization’s imperfect data with similar types of national data. Boards often prefer methods that are more quantitative because they are easier to manage; often the richest data outcomes can be developed from insights generated by more qualitative methods, based on small samples.
- Agree on specific indicators: Develop behavioral outcomes desired. For example, one of a number of specific goals might be “Some students find, as a result of cooperative experiences, that they made poor occupational choices.” Or, mentions in the local newspaper can be used as an indicator of public presence. There will often be temptation to add in other indicators simply because they are available, or because they “would be interesting to look at.” Keep the focus on the indicators of agreed-upon outcomes!
- Agree on judgment rules: Board and management need to agree at the outset upon the outcome metric numbers the organization would like to achieve for each specific indicator that contributes to the desired strategic objective. The rules can also specify values that are “too high” as well as “too low.”
- Compare measurement outcome with judgment rules: Determine how many of the specific objectives have been achieved to assess whether or not the strategic objective has been achieved.
These five points may be a challenge to agree upon. At a minimum this process requires time to meet together and discuss these points. This might best be done at a board retreat, perhaps have some experts come in and contribute research and information, or consider an outside facilitator to help guide discussion and resolution. These are topics of great passion and interest here at Brighter Strategies, let us know if we can be of any assistance. In our next blog we will continue to look at Fram and Talley’s research and the relational aspect of defining and using imperfect metrics.