IMPACT MEASUREMENT

Impact Measurement Done Right - Post COVID19 Guide 📚

IMPACT MEASUREMENT

Social Impact Measurement is a process of understanding how much social change occurred and can be attributed to an organization's activities.  Impact Measurement is not about measurement at all.  Real impact measurement is about integrating stakeholder voice to understand community outcome over a long time. 

COVID-19 pandemic and ensuing economic crisis have illuminated inequalities in society.  Both public, private sectors and philanthropy must come together jointly to scale the social impact of the community.  If applied properly impact measurement can be instrumental in scaling social impact. 

 

CURRENT STATE

IMPACT MEASUREMENT NEEDS PANDEMIC SHIFT NOT INCREMENTAL

Civic infrastructure creates a fundamental shift in the way communities work across systems to get better outcomes for youth and families.
JENNIFER BLATZ
President and CEO, StriveTogether
 
But how do we measure better outcomes?
  • Corporates realize that engaging stakeholder is not negative.  By engaging social and business value, they can create shared value for all stakeholders.

  • With the US election over impact investment and climate change will get a strong jolt.  This has already put a strong emphasis on ESG and Impact Reporting.

  • Increase social challenges, and limited public funding have put pressure on philanthropy to focus on social return on investment to justify the program's social and financial benefit and its effectiveness.

Big questions everyone is faced with are -

  • How do they accurately report and ESG and Impact?  Are they the same?
  • How do I know what social impact to measure?
  • How can we scale the social impact?
  • What is scaling social impact mean?
  • How can I effectively understand the social return on investment and compare different programs or investments?

Organizations often need impact measurement to understand many things related to stakeholders. 

  • Design effective product or services
  • Access market opportunities
  • Develop stakeholder insight
  • Improve Stakeholder engagement
  • Identify and mitigate the negative impact.
  • Measure and report to external audiences
  • Enhance and scale impact
  • Monitoring program progress
  • Predict long term impact

Scaling social impact cannot be equated to business or technology scaling.  Scaling has a different meaning in the social sector. Given a complexity such as a language, culture, and community needs, the same type of business scaling is not applicable. When we say "scaling," it means that a leading organization improves outcomes through better stakeholder listening and learning.

If they consistently demonstrate better outcomes over the years, other larger players such as the public sector or public/private can come in replicate based on results achieved by the leader. Many documented use cases such as mother and child mortality, employment for people with disabilities, and blindness exist. But even replication cannot be taken for granted as so many cultural barriers can make it difficult to replicate. All of them required a reference design followed up by others willing to apply locally.

Challenges of scaling social impact

Organizations that usually focus on capacity building provide training to service providers, volunteers, and other organizations. Their focus may be on mental health, grief support, disease prevention, suicide prevention, or public health such as palliative care. These organizations have a tremendous opportunity and obligation to learn from their programs. Traditionally these organizations use a survey-based pre-and-post analysis approach. However, traditional survey management tools can be inefficient with consistent data collection (multiple data sheets) and provide limited stakeholder understanding intelligence. More importantly, the analysis of results outside survey management can be time-consuming, error-prone, and delayed. There is also a danger of stakeholders drop off with the same questions multiple times and long surveys.

Challenge of monitoring and. evaluation

 

 

Impact justification barrier to scaling impact

Measuring and evaluating is not new!  The philanthropy sector has been doing that for decades.  Depending on challenges, different techniques have been adopted in Monitoring, Evaluation, Research, and Learning (MERL). Organizations use Randomized Control Trials (RCT), Monetization/Social Return on Investment (SROI),  or Lean Data. While there is merit to each of these approaches, they all are backward-looking, often focused on impact reporting or impact justification, and NOT continuous learning and improvement.

The concept of measurements, such as Net Promoter Score (NPS) and product/customer success analytics, has been successfully used in the corporate world. So, why does the social impact sector lag behind when it comes to using similar concepts? Bringing stakeholders' voices into the mainstream economy is critical for improving the social sector's efficiency. Often, today's technology focuses on the top of the pyramid - 20% of stakeholders. What are the challenges, and what is the right approach in understanding stakeholder want in terms of social change?

While many, especially from impact investing, tout the benchmarking approach for better investment and predictiveness. This approach often comes from the Wall Street mentality. They often miss the point that finding a reliable data source with the right context is challenging in the social sector.  On the other hand, the impact rating approach also has limitations. It misses the context of each enterprise's impact by using standardized questions and analysis.

Approaches like Lean data are promising but have limitations as it requires right technology to drive better results.  Lean data requires expensive data collection through an enumerator and provides a snapshot of data and NOT continuous learning data. Their analysis is often based on proprietary benchmarking, which may have the same benchmarking challenges described above. Most important, without continuous learning by front line managers, we cannot scale social impact.

Focus on monetization and monitoring, and evaluation often demonstrates the impact or justification of funding of the program. This precise goal is a significant barrier to improving stakeholder impact. Stakeholder impact requires continuous learning and improvement from the stakeholders.

Impact Measurement Challenges

The right approach measures the right social impact through continuous data collection and improvement in stakeholder outcomes.


Missing impact in impact strategy

Social Impact is all about the change that you create for a stakeholder.  If stakeholders are not integral to scaling social impact, funders or organizations are impact washing, SDG washing, greenwashing, or, as some say, rainbow washing. Impact is about scaling and not claiming how many children you fed or how many women they trained?  If an enterprise does not effectively use an outcome-driven approach, you are impact washing.

ASSET MANAGER (IMPACT FUNDS, GRANTMAKERS ETC)

IMPACT MEASUREMENT CHALLENGES

While each asset managers are in a different impact management phase, by enlarge, we still see that most asset managers design a framework that collects higher-level results from company or nonprofits.  They all have a different approach to collecting results, but very few actually work with downstream enterprises to improve or scale social impact.
  • Cherry Picking Metric
  • Extrapolating Results Without Involving Enterprise
  • Just collecting financial, governance, activity, or output data.

While there is significant progress in impact frameworks such as impact management projects, IRIS+, etc., many organizations still cherry-pick their impact measurement & management metrics. Often, there is a limited understanding of outcome measurement methods. Our experience has shown that there is still a large gap in organizations developing well throughout the impact measurement and management process, making it impossible to derive better fruits.

Often asset managers and corporations aggregate social and environmental results from external sources or partners. It can be operational output results (due diligence), such as large corporations aggregating carbon-related data from supply chain partners. 

As impact investors invest in companies with different impact thesis, they struggle to define the theory of change for each investment. Hence they fail in collecting results from investees. While some social impact-first try to go deeper, they still do not build capacity with investees on measuring social impact due to lack of impact intentions and dedicated impact accounting budget line item.

 

A typically matured funder when they collect results from their investee or grantees; each has its own proprietary data sources, programs that may not even keep track of impact specific to each funding cycle. As a result, aggregated results may not provide better results. Even if they do, it rarely understands stakeholder outcomes.

CSR and Community Foundation often struggle to aggregate impact as their funds are often limited or lack a long-term approach. These organizations should take a cue from some promising and emerging models. 

Focus on Stakeholder Impact

Real Impact can only be achieved when the asset manager becomes a partner to impact themselves and not remain a check-writing agency!   Often asset managers do not take their social impact role seriously. Building a deep impact portfolio requires that funder rolling up their sleeve and understand what it takes to create a true impact. Perhaps investee in few but effective enterprises focus on deep impact or work with a philanthropic partner to create deep impact.  Funders should design top-down and bottom-up impact management.  More importantly, they must realize that real impact will be realized when enterprises have the right strategy, data collection, and analytics to manage the impact.

ENTERPRISE WITH IMPACT INTENTION BUT LIMITED RESOURCES

An enterprise that has a strong social impact alignment but does not have impact management capacity
ENTERPRISE OR NONPROFIT

Step by Step Capacity Building

Most social enterprises or nonprofits feel that they are catalysts for creating social impact.  However, they are neither clear about what the specific impact is and clearly cannot demonstrate evidence.  Without often, they fail to scale as often investors demand evidence.  Often these organizations are challenged with business and financial constraints.  The last thing they think about is "social impact" due to a lack of resources.

How to break this logjam?

Building impact capacity is critical to help this organization scale. While social impact accelerators and impact first funds are naturally aligned to help these organizations, other funds should not get a free ride. If they provide investment capital without impact capacity-building, it's time to rebrand them as regular funds as they are not sincere about creating impact.

Step by step impact measurement capacity

  • The first and most important step is to build an impact maturity model. SoPact has been working with leading impact first funds to develop a high-quality survey aligned to five dimensions of the impact that comprehensively understands each company's impact management capacity. Some of the questions are  
    • Maturity of the "theory of action."
    • Design enterprise impact management capacity survey to assess the true capacity
      • Business & finance maturity
      • Understanding of true demographic (WHO)
      • Product or Services dimensions of Impact (WHAT, HOW MUCH, CONTRIBUTION, IMPACT RISK)
      • Capacity Building Workshops
        • Define enterprise theory of change by the impact theme. For example, inviting a pool of companies with the same impact theme, let's say a smallholder farmer. an organization working with coffee plantations, palm oil can use an out-of-box IRIS+ theme designed as a "theory of change." 


Modify the key outcomes to make them specific to the investment. For example, the above investment will with IRIS+ Small Holder Farmer  modified to meet internal goals.
    • Define stakeholder data enumeration approach. 
      Because each organization stakeholder is situated differently, enterprises and funders will need to brainstorm to develop the most optimal process. Impact Cloud™ does help you with data collection challenges.  Talk to our team to come up with the most optimal approach.
  • Getting ready for analytics: 
Impact Cloud simplifies many data collection approaches and prepares you for data analytics quickly.  The key question will be who will responsible for integrating data and building analytics? Initially, funders can help out, but ultimately prepare them for phase 2, where enterprises can take responsibility for continuous learning and improvement.
  • Build a real-time dashboard: Every enterprise should have an incentive to collect data.  The dashboard can help them motivate for an outcome-driven journey.

ENTERPRISE LOOKING TO BUILD CAPACITY AND SCALE

A large capacity-building organization that have been collecting stakeholder data but impact data management is complex and messy
IMPROVE IMPACT MEASUREMENT IN

CAPACITY BUILDING

Organizations that focus on the capacity building provide training to service providers, volunteers, and other organizations. Their focus may be building individual, organizational, network, or system capacity.  These organizations may be focusing on single issues or integrated issues such as mental health, grief support, disease prevention, suicide prevention, or public health such as palliative care. These organizations have a tremendous opportunity and obligation to learn from their programs.

These organizations may be using consulting, training, mentoring, information & referral, peer exchange & learning. Defining what ‘capacity’ means and long term outcome can be quite challenging. Depending on the objective, organizations may employ many evaluation approaches. Regardless of evaluation and analytics, the fundamental challenges that each of these organization are facing are data mess.  

Traditionally these organizations use a survey-based pre-and-post analysis approach. However, traditional survey management tools can be inefficient with consistent data collection (multiple data sheets) and provide limited stakeholder understanding intelligence. More importantly, the analysis of results outside survey management can be time-consuming, error-prone, and delayed. There is also a danger of stakeholders drop off with the same questions multiple times and long surveys.

IMPACT MEASUREMENT STRATEGIES

Listen and learn from Chris Gaines, Lead Trainer, complete impact measurement primer.

  • 01 Impact Strategy
  • 02 Impact Data
  • 03 Survey Design
  • 04 Data Analytics
  • 05 Impact Reporting

FOUNDATION OF IMPACT MEASUREMENT

The first step in impact measurement is a robust impact strategy. Whether you are a funder or an organization working in the field, you might want to align with multiple frameworks, such as IRIS, Impact Management Project, and Sustainable Development Goals. 

DATA COLLECTION AND AGGREGATION

Impact data is a foundation for continuous learning improvement.  As well designed impact data will design a clear strategy of data collection from different data sources.  A well-designed impact data warehouse can manage data aligned to impact strategy.

  • Activity tracking
  • Outcome harvesting
  • Partner activity and results
  • Outcome results
  • Performance results

 



Design Outcome Tracking

Outcome tracking is critical in improving collective or community outcomes.  Depending on the impact goal, impact practitioners can design an outcome evaluation process. Learn more about the most popular outcome tracking design approach from Chris Gaines, SoPact's Lead Trainer.

Evaluation Data

Integrated data warehouse and analytics are key to continuous learning and improvement.  Impact Cloud is a unique platform that allows in-built analytics with various techniques based on outcome evaluation goals.  Listen to Chris Gaines, Lead Trainer, SoPact describe the 8 most common analysis techniques.

Impact Reporting

Integrated data collection, data aggregation, and data analysis are critical to impact reporting and impact learning process. A well-designed system allows for quick alignment of evaluation goals and reporting metrics specific to different donor requirements. As you impact data and outcome tracking improves, it is important to design a system that clearly communicates results through all major impact dimensions. Listen to Chris Gaines, Lead Trainer, on how to build an effective impact reporting system.

TOOLS AND PLATFORMS

IMPACT MEASUREMENT ADVANCED TOPICS

Impact Measurement Norms for Powerful Results

  • How do you build an effective impact measurement system?
  • What is the current state of impact measurement and management?
  • Join us to learn from Jane Reisman and Veronica Olazabal, sharing how to integrate standards, use different frameworks for different sectors, and incorporate learnings from the data to make strategic decisions.
  • Learn how Impact Measurement, Impact Management Project, and IRIS+ is advancing end to end impact management.

 

Why Social Impact Measurement is a Contact Sport?

What is the social impact? How do you know what if you are scaling social impact?

Social Impact Measurement is not just about measurement. It is about the stakeholder's voice. This video is a live conversation with David Bonbright from Keystone Accountability. He is a champion of the stakeholder's voice (Constituent Voice) or Feedback Loops.

Social impact measurement is a process that is about continuous impact data measurement and improvement. Stakeholders or constituents must be at the front and center to scale the social change.

"The largest benefits related to improved measurement practices are those that captured the voices of an organization's constituents." - David Bonbright, Keystone Accountability, and Feedback Commons.

Keystone Accountability helps organizations understand and improve their social performance by harnessing feedback, especially from the people they serve. David will introduce and explain the underlying theory and design principles for Constituent Voice:

 

TOOLS AND PLATFORMS

SOCIAL IMPACT MEASUREMENT TOOLS

Many vendors classify themselves as social impact assessment, measurement, and management.  However, please note that there are significant differences.

Impact Rating & Benchmarking

B-Analytics, B-Assessment ≠ (not equal) Social Impact Measurement

First-generation impact assessment platforms focus on gathering governance, environment, and social policies to assess entrepreneurs or Small Medium Business (SMB) or Small Growing Business (SGB). Here are four examples:

1. MIX Markets: For Microfinance Institutes.

2. AERIS Cloud: For CDFI Institutes.

3. B-Assessment:  Impact on its workers, community, environment, and customers.

4. B-Analytics: For investors, supply chain managers, business networks, and governments in helping companies to measure and improve their positive impact.

There is a major theme behind these first-generation impact assessment resources. Their main goal is to provide benchmarking to the institutes that they focus on. The problem is that they try to use "impact assessment" with a broader brush.  The reality is that impact is context-sensitive and cannot simply be measured through ratings!

Limitations of Self Reporting

Tools like Clear Impact allow public agencies to create an impact scorecard.  While scorecard-based reporting is useful in public agencies where regulatory reporting is important, their value remains limited. 

Fundamentally, the social impact should be driven and managed throughout a program's life cycle based on a well-defined theory of change or logic model.  Tools like Clear Impact require you to define your own custom metrics and do self-reporting.

Unfortunately, reporting organizations can use any data collection system, making them less audit-able with limited integrity and limited impact context.  Non-existent data management limits an organization's ability to acquire and learn from results.

Customization-Based Systems

Many legacy custom applications require software vendors to modify business objects. Hence, the user must rely on the software vendor to customize the code based on a programmatic approach. .NET, Java, or legacy client/server applications fall into this category.

 

Social Impact Software Customization

Cloud Platforms such as Salesforce/.NET 

While systems like Salesforce CRM have the capability to configure, each Salesforce instance must be configured by expensive Salesforce developers (either in-house or by an external consultant). For example, if you want to upgrade from IRIS 4.0 to IRIS 5.0, you must rebuild your metrics catalog.

There are advanced services possible but building a truly user-configurable system that is flexible takes much effort. Most social sector organizations do not have the capacity to build out such software or keep up with ongoing changes in requirements.

True impact assessment & measurement systems require an immense amount of flexibility, are the theory of change driven, allows rapid integration of data collection of any kind, and are optimized for quality reporting.  Such a system takes a long time to build, and most social sector organizations do not have enough resources, skills, and alignment between the IT department and program management teams to do so.

Social Data Solutions company like Vera Solutions provide a customized Impact Assessment & Measurement Tool. However, while they start with a packaged approach, the reality is that every organization's theory of change, indicators, data collection, impact storytelling, and learning and impact reporting are unique.  Things that you should consider before evaluating such systems are: 

  • What is a total cost of ownership (cost of subscription + upgrades + maintenance)
  • Will the system be flexible to meet different grantee or investee requirements, which will undoubtedly change over time?  How will you make changes?
  • Will you get all the list of features in a comprehensive package?
  • How long will it take to deploy?
  • What's the training time and cost?

Configuration Driven ≠ (not equal) Customization

Beware of systems that require customizations.

Impact Cloud is designed so that any user can configure business rules without any technical knowledge. It is at least 10 times faster to implement than Salesforce and easier to maintain.

Impact Measurement System must be...

  • Flexible
  • Comprehensive
  • Easy to use
  • Rapid deployment with easy to configure rules or metrics changes 
  • Theory of Change driven.
  • Flexible enough to adapt to each organization's use of standards, framework, and tools
  • Work well with any types of data collection tool that enterprises and nonprofits use.
  • Simplifies data aggregation and unlocks value and story of data

CONCLUSION

Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. Lorem ipsum dolor sit amet, consetetur sadipscing elitr. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua.