More Imperfect Metrics
We are examining the research of Talley and Fram in the October edition of Non-Profit Quarterly that allow for using imperfect metrics to measure and inform strategic objectives. The following example illustrates another example that likely applies to your organization:
Families Primary is a nonprofit counseling service offering a range of services to improve mental health in the metropolitan community in which it is located. Services range from individual counseling to being legal conservators for elderly clients. The mission of the organization is to reduce mental health problems in the community. Local county health officials have noted a significant increase in inner-city mental health problems. However, the use of the agency’s services by these residents was very modest. The costs of conducting a reasonably comprehensive client attitude and mental health needs assessment study would be too high. Yet not measuring these key outcomes leaves the agency vulnerable to any number of criticisms.
The president/CEO was evaluated by a board assessment committee. Committee members took primary responsibility for establishing board-approved goals and evaluating specific outcomes (for examples, finance, personnel, and fund development) of the operation. The person assigned to client development was asked to create outcome measures for improving the perception of Family Primary among inner-city residents. He and the CEO concluded that a board member needed to interview the executive directors of five inner-city community centers to obtain a macro-assessment of the agency’s images. Cost constraints prohibited developing more precise outcome metrics.
All five executive directors unanimously reported the local residents were “uncomfortable” with the agency’s staff.
Based on the interviews’ outcomes, the board member and the CEO agreed that in 12 months, the board member would revisit the five executive directors to assess changes in perceptions, based on corrective actions to be instituted by the CEO. It was the responsibility of the CEO to devise the corrective actions that were needed to drive change. Subsequently, a quantitative goal would be set for recruitment of inner-city clients.
The next year, the executive directors reported some improvement in perceptions. However, it took a second year of corrective actions before the board member and the CEO agreed it was time to establish quantitative outcomes to evaluate performance. Incremental achievement had taken place driven by an imperfect outcome measurement.
The metrics were admittedly qualitative, subjective, and vulnerable to unpredictable distortions; in short, they were imperfect. But they provided a focus on relevant issues. They linked players into productive conversations with each other about where to spend resources and when to revisit their progress. There was a process robust enough to benefit from poor metrics.
Might your organization benefit from such a process? Have you attempted such a process already? Let us know your experience with metric gathering and interpretation.