There's a common Catch-22 in the social impact assessment or social impact measurement field.
Funders often choose metrics based on the varying capacity of grantees. Paired with the limited data-capacity faced by the funders themselves, this results in assessments which only capture the low hanging fruit of the data world.
In other words, it translates into the selection of easy-to-measure activity- or output- focused metrics.
In this article, we'll further explore why this tendency is so pervasive, how it works against the goals of a social impact assessment, and what we can do about it.
Whether you find yourself on the funder side, the asset side (social enterprises, etc.), or somewhere in between, by the end of this article you'll better understand how to avoid the primary social impact assessment pitfall, and will be armed with the tools necessary to do so successfully.
Why Your Social Impact Assessment Questionnaire is Failing
A social impact assessment ostensibly tells us how well a program or intervention is performing in terms of achieving impact outcomes.
And yet, our view is that most assessments fail because they too often focus on outputs that have occurred as opposed to getting at outcomes.
Outputs are, of course, easily measured. Your questionnaire for program managers or beneficiary liaisons might include questions like: How many hours of training were delivered; How many people were served; How many products are being used by beneficiaries. These imply impact, but do not demonstrate it.
And these metrics hold little strategic value for the grantees, especially if they are only measured for a short period.
And so, a stigma has evolved around the trustworthiness of the data received as well as how helpful what we're calling 'measurement and evaluation' really is to discern the impactful results of our interventions. With such little regard for the practice, there is little incentive to invest in improving the process, and so, here we are in a Catch-22 of activity-driven (i.e. output-driven) measurement.
Is Outcome-Driven Social Impact Assessment Worth the Effort?
Something wonderful happens when an organization decides to have an outcome-driven social impact measurement process. They begin to reassess their impact framework more critically than ever before. That's because, to build an enduring practice for impact measurement, an organization must have a well-developed impact foundation to guide it through the metrics selection. This starts with their Vision statement and moves through their Theory of Change.
What we've found is that some of our clients were missing elements of this impact framework. The ability (for the first time) to measure for results facilitates an internal conversation around the broader impact hypothesis, and the organization is left with more established impact messaging.
The benefits of results-driven evaluation go beyond the structuring of an organization's impact framework. It includes the strategic insights gained from a better understanding of the outcomes of your interventions. Armed with such insights, your organization is better able to iterate the process. And when built upon a solid foundation that includes a thoughtful Theory of Change, an organization is more aware of its assumptions and areas of potential risk. With an outcome-oriented approach, measurement and evaluation become a pivotal element of your organization's strategic decisions. With the ability to more confidently speak to the specific results of your interventions, your marketing and donor relations will also benefit.
Improved Social Impact Measurement Processes
One phrase that pops up again and again in the unfortunate cycle is "data capacity". If we might be able to enhance our own capacity and that of our partners, perhaps we will be able to take on more challenging metrics that offer more strategic insights. Once the value of the data becomes apparent, its positive impact on programming, marketing, donor relations, and grant seeking lead to better staff engagement with the data reporting. Especially if reporting becomes more user-friendly.
Impact Cloud™ was designed to be accessible and thorough, flexible enough to meet the data needs of various socially minded organizations. And it starts with a well-developed impact framework. Actionable Impact Management begins with the introspective work of defining a Vision, Mission, and Goals; Program Structure; and Theory of Change before moving on to the selection of metrics and your data strategy.
The transformation from activity or output metrics to outcome metrics can mean all the difference in buy-in. By strengthening data capacity, funders can make that transformation a reality.
Read More: How can we improve social impact accounting?
Written by Rachel Dodd
Rachel leads the Actionable Impact Management (AIM) framework development from SoPact's side and oversees partnerships.