<img height="1" width="1" style="display:none" src="https://q.quora.com/_/ad/0609320ad13c4183af7978b1c1744b71/pixel?tag=ViewContent&amp;noscript=1">
Measuring Social Impact

The Best Practices on

Measuring social impact

A complete Primer for Social Businesses, Nonprofits & Enterprises!

What is Impact Measurement

Social Impact Measurement is a process of understanding how much social change occurred and can be attributed to an organization's activities. The integrity of the process is usually bolstered by an impact statement and a clear impact-driven, stakeholder-focused strategy.  It  is very different from data collection, impact assessment and impact reporting but often gets mixed into those categories! The following is a comprehensive guide to help give you better clarity on what it is, and what it may mean for you and your organization as you delve into the different approaches and best practices.

Introduction

Impact Measurement  is an evolving practice used by mission-driven organizations, sustainable development organizations, for-profit businesses, impact investors, and more. 

Lack of proper funding, training, and the right technology adoption across sectors have traditionally limited its true value-creating potential.  While the rise of impact investing & other trends in philanthropy is reducing some of these headwinds, we are still a long way of reaping the true benefits of evidence based approach

To get started:

  • Let's look at the definition from Forbes
  • Let's learn how evidence based approach helps raise smart capital through faster evidence 
  • Let's learn different approaches, common misunderstandings and how to optimize value 

Getting Ready for Social Outcome Management

It Starts with the Theory Of Change

Social Enterprise, nonprofits, and government organizations require a well-defined theory-of-change-driven cycle of impact creation.

The key steps for creating impact value are:

  • Theory of change 
  • Social Impact Indicators
  • Impact Thesis
  • Data Strategy
  • Data Collection
  • Data Aggregation
  • Data Analytics 
  • Impact Learning & Reporting

Let's dive into a few of these.

Social Impact Data Collection Is One Part Of The Puzzle!

Excel, Google Spreadsheets ≠ (not equal) Social Impact Measurement

Excel is good startup tool with Many Limitations

Most organization prefer to use data collection tools based on their internal knowledge, data capacity and cost.  While there are hundreds of data collection tools, in general these tools can be be largely classified.  The most popular tools are paper and excel (or Googles Docs).  In fact, Excel and Google Spreadsheets are arguably the most used tools today for organizations attempting to measure impact.  

But........

Excel was never made for social impact measurement!

excel impact data aggregation

 

While Excel or Google Spreadsheet are well-known and relatively easy use, they cause many pain points as organizations grow and require more advanced data needs.

Challenges:

  • With many stakeholders and more and more data, this rapidly creates data islands, making it difficult to aggregate and analyze results in integrated manner across partners, departments, and teams.
  • Not easy to track data sources
  • If metrics change, it is difficult to analyze and report in an efficient and accurate manner
  • Data can be hiding anywhere especially when organization have high turnover

 

8 Best Practices of Impact Measurement

 

Social Impact Evaluation & Measurement Tools

Mobile Offline Data Collection!

For Rural or Difficult to Reach Beneficiaries 

offline data

For measuring outcomes and satisfaction level of beneficiaries in remote areas mobile offline data collection tools play a greater role. There are hundreds of ODK and non-ODK (HTML5) based mobile applications, although there are some key limitations you should be aware of.

Challenges:

  • Survey tools do have basic analytics but are often limited to specific batches of data. Analyzing results over time and aggregating with other external results brings similar challenges those with using Excel as described in the previous section.
  • Not easy to track data sources
  • If metrics changes, it is difficult to analyze and report
  • Data can be hiding anywhere especially when organization have high turn over

Learn More:  Online & Offline Data Collection

Impact Assessment Survey Best Practices

For Difficult to Reach Stakeholders 

Young populations, rural populations or day laborers in factories are not easy to reach, but they can often be reached through SMS- or a Voice-based approach. There are many modern technologies such as Twilio which deliver an API-based approach, combining effective communication tools. 

For Online Users

Tools like Survey Monkey, Qualtrics, Google Forms, and online database spreadsheets like Airtable are widely-used tools for collecting online survey results from stakeholders and beneficiaries.

Each delivers on a unique value proposition while also fundamentally serving the purpose of recording data. Once the data arrives, however, it is up to the data collector to analyze, build useful reports and tell compelling stories.

The main value of these tools is often to understand product satisfaction, net-promoter score (NPS) or even understand outcomes. However, survey tools or survey-based measurement (often packaged by grant management systems) should not be used in place of a flexible measurement system. which leverages other tools to greater effect.  

 

5 Things to Consider For Impact Assessment Survey

survey tools for impact measurement1. Surveys are just one piece of a puzzle

Often, surveys unveil just a slice of data, just a piece of the full insight you can be garnering through your impact measurement efforts. True social impact demands a hybrid approach and depends on the impact thesis of an organization or program. Ask yourself:

  • Is our ecosystem investment based on outcome alignment with investors?
  • Is program investment based on program effectiveness (efficiency in implementation)?
  • Is program investment based on demonstrating social change?

An impact thesis helps an organization define a strategy which covers operational capacity, scorecards, stakeholder assessment, and general alignment of required criteria that demonstrate risk and effectiveness.

Often organizations have to develop custom analysis processes to bring all the data together to understand program effectiveness, social change, and outcome alignment.

Without an integrated analysis, superficial survey data can pose a danger to data integrity.

2. Survey vs. Results Collection

Surveys are useful in gathering one-dimensional data from stakeholders, such as baseline data, and pre and post feedback data. This process should not be confused with collecting results. At a high level, there are two types of results-gathering from organizations or partners:

1) Output and outcome-oriented results 
2) Process-based results with or without a scorecard

While the first allows for understanding of overall outputs & outcomes of an initiative, the second is relevant for understanding program effectiveness and risk.

Often organizations use well-defined standards or custom metrics/indicators to collect results. A simple survey cannot collect RESULTS.

3. Better data analysis requires better relationships

Social impact results-gathering and analyses tend to be unique to each organization. High precision results can be achieved when metrics are assigned to programs based on logical relationships.

However, it is up to you to design your data structure, often done using tools like Excel which unfortunately do not represent well the relationship with Program, Partner, Product, Project, Applicant, etc. 

impact dashboard

A well-designed impact management platform (such as the Impact Cloud shown above), on the other hand, allows you to collect data from the online or offline surveys and index them in the database for a future reference. This will ultimately help streamline your reporting without creating islands of data stories. 

4. Pitfalls of using irrelevant systems for impact management

Many of the grant management software solutions available include impact measurement applications. Unfortunately, most of them enable only the survey-like data gathered from grantees/investees.

Survey respondents are likely to respond based on a size of an investment or based on local compliance requirements before providing true results to the funder. A funder-designed the survey is likely to to be limited to assessing solely for program objectives.

Usually, grantee impact is unique and survey questions are not always relevant to them and their impact context. Other times they simply don't have the necessary data capacity either (a system which tracks activity, outputs or outcomes).

Grantees may also feel that they have already provided answers in a similar survey (frequent survey syndrome) and believe that another survey with similar questions does not serve a useful purpose. 

5. Generic Surveys fail to understand deep impact

To understand actual impact risk you must aggregate results based on an integrated stakeholder approach.

For example, if a foundation is funding selected non-profits and wants to understand if recipient organizations are improving volunteer capacity or not, survey tools can measure volunteer management but then have to be compared against an indicator like volunteer opportunity.

Once results have been acquired, all the data manipulation usually happens in a tool like Excel, with customized pivot tables and data analysis. As the years and number of organizations increases, an Excel-based approach becomes increasingly messy and unmanageable.

In other words, it is difficult to leverage just survey data to tell a compelling impact story. What do you need to tell an effective story? We recommend:

  • Collecting data from partners and beneficiaries/clients that are relevant to them.
  • Aggregate data in such a way that offer unique impact insights to programs, partners, or projects with logically similar entities.
  • Provide consistent and easy to use impact insights for different types of data aggregation in a single location.

Case Management & Customized Database Applications

Many social sector or human resource agencies use case management processes to track beneficiary data, incidents and program data.  Often these systems are customized to meet each unique use case of an agency in areas such as homelessness, disability, etc. Make sure not to confuse these approaches with impact reporting. 

Some of the challenges are:

  • Merely report activities which not occurred, not the impact they may have (or haven't) generated.
  • Difficult to align with the unique requirements of many funders
  • Reporting by different frequency (monthly, quarterly, annually)
  • Aggregate results between multiple programs
  • Outcome management

Evolution of Impact Assessment & Measurment Tools

There are many vendors which classify themselves as social impact assessment, measurement and management.  However, please note that there are significant differences .

Impact Rating & Benchmarking

B-Analytics, B-Assessment ≠ (not equal) Social Impact Measurement

First generation impact assessment platforms focus on gathering governance, environment and social policies to assess entrepreneurs or Small Medium Business (SMB) or Small Growing Business (SGB). Here are four examples:

1. MIX Markets: For Microfinance Institutes..

2. AERIS Cloud: For CDFI Institutes.

3. B-Assessment:  Impact on its workers, community, environment, and customers.

4. B-Analytics: For investors, supply chain managers, business networks, and governments in helping companies to measure and improve their positive impact.

There is a major theme behind these first generation impact assessment resources. Their main goal is to provide benchmarking to the institutes that they focus on. The problem is that they try to use "impact assessment" with a broader brush.  The reality is  that impact is context sensitive and cannot simply be measured through ratings!

Limitations of Self Reporting

Tools like Clear Impact allow public agencies to create an impact scorecard.  While scorecard-based reporting is useful in public agencies where regulatory reporting is important, their value remains quite limited. 

Fundamentally, social impact should be driven and managed throughout a program's life cycle based on a well-defined theory of change or logic model.  Tools like Clear Impact require you to define your own custom metrics and do self-reporting.

Unfortunately, reporting organizations can use any data collection system, making them less audit-able with limited integrity and limited impact context.  Non-existent data management limits an organization's ability to acquire and learn from results.

Customization-Based Systems

Many legacy custom applications require software vendors to modify business objects. Hence, the user has to rely on the software vendor to customize the code based on a programmatic approach. .NET, Java or legacy client/server applications fall into this category.

 

Social Impact Software Customization

Cloud Platforms such as Salesforce/.NET 

While systems like Salesforce CRM have the capability to configure, each Salesforce instance must be configured by expensive Salesforce developers (either in-house or by an external consultant). For example, if you want to upgrade from IRIS 4.0 to IRIS 5.0, you must rebuild your metrics catalog.

There are advance services possible but building a truly user-configurable system that is flexible takes a lot of effort. Most social sector organizations do not have capacity to build out such software or keep up with ongoing changes in requirements.

True impact assessment & measurement systems require an immense amount of flexibility, are theory of change driven, allows rapid integration of data collection of any kind, and are optimized for quality reporting.  Such a system takes a long time to build and most social sector organization simply do not have enough resources, skills and alignment between IT department and program management teams to do so.

Social Data Solutions company like Vera Solutions provide a customized Impact Assessment & Measurement Tool. However, while they start with a packaged approach, the reality is that every organization's theory of change, indicators, data collection, impact storytelling and learning and impact reporting are quite unique.  Things that you should consider before evaluating such systems are: 

  • What is a total cost of ownership (cost of subscription + upgrades + maintenance)
  • Will system be flexible to meet different grantee or investee requirements which will undoubtedly change over the time?  How will you make changes?
  • Will you get all the list of features in a comprehensive package?
  • How long will it take to deploy?
  • What's the training time and cost?

Configuration Driven ≠ (not equal) Customization

Beware of systems that require customizations.

Impact Cloud - Impact Maker  is designed in such as way that any user can configure business rules without any technical knowledge. It is at least 10 times faster to implement than Salesforce and easier to maintain.

The best Impact System must be....

  • Flexible
  • Comprehensive
  • Easy to use
  • Rapid deployment with easy to configure rules or metrics changes 
  • Theory of Change driven
  • Flexible enough to adopt to each organization's use of standards, framework and tools
  • Work well with any types of data collection tool that enterprise and nonprofits uses
  • Simplifies data aggregation and unlocks value and story of data

Acumen Lean Data Measurement and Lean Impact Approach

Why use a lean data measurement approach ?

Acumen has popularized the lean data measurement approach for social enterprises. Recently, Ann Mei Chang has been evangelizing Lean Impact Book in a nonprofit context.

While these are great starting points to understand a lean approach, most of these organizations still need better tools or platforms which enable true social impact measurement & management with a lean data approach.  

The best lean data platforms:

  • Go beyond the mere data collection (SMS, Offline or Online) often prescribed 
  • Must be theory of change driven with a well-defined impact indicator aligned with theory of change
  • Easy to use and help find the needle in haystack (hidden impacts) as most organization don't have a data scientist in team.  Hence, tool should allow for faster impact learnings and feedback
  • Report and communicate impact effectively to raise capital based on evidence of impact

 

 

Data Aggregation from Different Data Sources

As organizations grow so does their use of data management systems.

Organizations who have to report to foundations or public agencies may have unique reporting requirements that must be met. The challenges in such situations:

  • Organizations may not be collecting all the reporting requirements
  • Separate outcome management may be required
  • Data extraction, and aggregating into individual spreadsheet can be time consuming and tedious
  • May take a long time to create individual reports

 

social impact data and reporting process

Unlock a story from collected data

Most organizations have their own operational, program, beneficiary, stakeholder and financial data management systems.  Reporting for different funders to meet their own unique reporting requires the aggregation of results from different data stores.

Some example data collection systems:

  • Paper-based
  • Excel or google spreadsheets for program  or operational data
  • Mobile Offline data collection for beneficiary survey
  • Custom Case management and program management system.
  • Customized databases
  • Salesforce or similar systems
  • Online survey tools such as Survey Monkey

While most organizations will continue to track data using their favorite approach, reporting consistent results and tracking past results is challenging. With growth, the entire data system can look more like data islands.

Data Warehouse

The best approach is to provide a simplified data warehouse that allows organization to  intelligently map data from any data sources and store them in a cloud-based database.  Once intelligent mapping is defined, field managers and operational staff can simply drag and drop results to the cloud every time new data and/or results are acquired. 

Measuring Outcomes Of Program or Project

Improve data management system

Good reporting requires first an analysis of our as-is (current data) situation. This requires looking at key indicators and understand reporting requirements.

Based On Reporting Period:

  • Annual
  • Quarterly
  • Monthly
  • Over a specific period of time

Based On Reporting Type

  • Total Count (Number)
  • Percentage
  • Unique Count

Two Dimensions

  • By Options

Complex Reporting Type

  • Formula based
  • Aggregate from different projects/programs
  • Compare results with different project/programs

Outcome Measurement

A Theory-of-change-driven outcome is the best feedback loop

Most social sector organizations and businesses focused on social outcome must have a robust theory of change. Limiting yourself to measure activity and output based social impact, gives you only a partial view of your true success.  Organizations working to verify their outcomes must build robust outcome measurement practices.

Through a true iterative and lean data approach, organizations can use this process to dramatically improve outcomes and impact.

Key design principles are:

  • Survey design - effectiveness of survey that gives deep insights into your beneficiary
  • Frequency and method of survey
  • Authenticity and trustability of survey
  • Key indicators and data collection aligned with them

How to measure impact of early-stage social organization

You must start an impact journey somewhere

Lack of data is not a problem, but lack of impact strategy is!

Many social organizations often blame the cost of measurement as a main hindrance to its implementation. The reality is that it is not cost but intention that is hindering progress.  While we believe that social outcome & impact measurement may not be appropriate or difficult for some initiatives, when implemented it must have a clear strategy behind and a well-built structure supported by the right tools.

Remember:

  • Early stage organizations can use a well-defined impact strategy to raise capital even without data
  • They must have a clearly defined
    • Impact Strategy
    • Theory Of Change or Logic Model
    • Impact Statement
    • Output & Outcome Indicators
  • Alignment with Impact Management Project can certainly help align impact thesis
  • Don't wait!  Start your impact journey now with Freemium Impact Cloud.

Reasonable Data & Robust Storytelling

Raise impact capital with good evidence

The key is to build a robust data-driven culture

You are successful social enterprise or mission driven organization. Impact investors are recognizing your story but they need more well-defined impact communication or evidence.  This is why you should seriously consider building a robust data-driven culture

Remember:

  • Reevaluate your theory of change and data culture
  • Make an inventory of all the data from past
  • Find data gaps based on revised theory of change
  • Invest in robust data management and aggregation platforms
  • Build a clear impact communication strategy
  • Start your impact journey with Impact Maker.

Creating Impact Data & Impact Learning culture

Create a culture of impact learning

Robust lean data management will save expensive mistakes and time

You are successful social enterprise with a decent data collection experience.  You are ready for growth in new programs, services or product.  Build a robust culture of lean impact/lean data to ensure a rapid feedback loop from your beneficiary. The key is to reduce longer monitoring & evaluation time & cost and implement the program with a rapid feedback from beneficiaries. 

Remember:

  • Build a consistent theory of change driven data strategy
  • Start with your data collection strategy before program start
  • Gather results with a limited beneficiary and survey sample
  • Learn from outcomes and improve your strategy
  • Keep doing this iteratively
  • Start your impact journey with Impact Maker and engage SoPact team or partners for robust strategy.

Measuring nonprofit social impact

Data Management and Analysis in Management

Is Important but must be simple 

As nonprofit organizations grow their data, events, fund raising, and program data applications grow.  As the number of data sources increases, nonprofits find themselves encountering major challenges.

Nonprofit data management and analysis will often find:

  • Uncoordinated data islands
  • Tracking data 

Unfortunately, many consultants prescribe the solutions more aligned towards the corporate world. The big challenge to this approach is that for each obstacle there are different tools and most nonprofits do not have technology skills and knowledge to integrate different tools.

The best solution must rely on simplified and intelligent field mapping between source data and a cloud database application.

Impact Measurement for Nonprofit Startup Funding

As fundraising becomes more competitive, and more Millennials engage in philanthropy, funders need to rethink their approach and investment thesis for fund allocation. In fact, we are seeing evidence of many foundations starting to evaluate impact investment as a vehicle for a long-term social impact.

Using Outcome Insights to Improve Non Profit Data Integrity

Today, most non profit organizations measure impact in a limited way: with inputs and outputs. The difficulty of finding outcome insights from grantees can create a cycle of mistrust, as funders are not confident about long-term social change.

While Randomized Controlled Trials (RCTs) have been seen as one popular vehicle to measure outcomes, many funders are hesitating to use it because it’s slow and expensive to run them. I suggest that non profits develop a systematic outcome measurement process to collect data to measure baseline and endline results. While that may seem daunting, a “lean data” approach can provide some helpful insights into beneficiary and community outcomes.

Impact Measurement in Foundations

One of the most effective actions a foundation can take to become a steward of change management is to guide impact discussions, facilitate outcome definitions, support nonprofit activities, and evaluate change management processes, all of which are necessary steps in developing nonprofit organizations capacity for change.

Foundation board members can actively engage in facilitating change management by operationalizing impact in a way that is meaningful to their grantees; leadership is essential as nonprofit organizations learn and practice change management. They can drive a focussed initiative for scalable social change like climate

Measuring Social Enterprise Social Impact

Why Measure Social Impact

Raise investment capital from investors aligned with impact thesis

Every organization must fully understand their current capacity and goals for social impact measurement. Learn more about why social enterprise should measure here. In general, before you start on the impact measurement journey, examine your goals and current state!

  • Do you have no data, limited data or reasonable data?
  • Do you have an effective pitch to align with impact investor aligned with your impact thesis?
  • How can you effectively demonstrate your impact if you have limited data?
  • How can you raise more impact capital if you have a good impact story to tell?

Social Impact Measurement Resources, Tools & Frameworks

Sign up for Impact Cloud® Managed Trial
New call-to-action
Jump To Section:
Join Social Impact Network of Impact Practitioners, Social Entrepreneurs, and Investors
Deep Impact Network
Social Impact Measurement

Stay ahead of the game, subscribe weekly!