Are you a mission-driven organization looking to build impact data strategy, where do you start? This article gives a practical and step-by-step process that you can follow based on your organization's stage. The purpose of this article is not to show you a finish line, but to demonstrate how to take a step-by-step journey. The truth is that most organizations remain at an early stage, rather mission-driven organization should build a long-term iterative strategy to continuously improve and come closer to your mission and vision.
The key benefits of taking this approach are:
- Improved understanding of your impact roadmap to meet the purpose of your organization
- Real knowledge about the effectiveness of your programs, products or services
- Access to impact capital based on the most effective use of impact data
- Making your role even more meaningful
1. Impact management starts with impact strategy
Let's say that your organization is just starting with the first impact report, where do you start? Starting from the data is not the right way. The foundation of high-quality impact reporting always begins with an impact strategy. So the next question is, how do I define my strategy? Depending on your impact perspective, you can start with the theory of change, logic model, logical framework or the Impact Management Project dimensions. And based on that, design the indicators to measure performance.
- Create a impact strategy per project, program, country, or portfolio
- Improve the understanding of the context by defining risks and assumptions
- Build on feedback-loops by relating outcomes to the activities for a more efficient impact management
- Develop a due-diligence and application process aligned with the impact
2. You have some program and operational data, now what?
Start with a clear documentation of all AS-IS data
Chances are that your first inventory will reveal that you already have a some of the following data:-
- Financial Results: Grants, revenue, expenses, other accounting results
- Operational or Activity Results: Number of students, trainings etc
- Stakeholder or Beneficiary Demographics: race, income, age etc
- Fundraising reporting: Donor profile, donations results
The first step here is to build a clear inventory and examine data. As soon as you have a created the inventory, you are likely to see incompleteness or duplication of data. You must allocate sufficient time to cleanse data once and for all. While this data can serve as a baseline for you, you will need to prepare a strategy to avoid a frequent data aggregation pain in every impact reporting cycle. Next do this -
- Remove any duplicated results
- Do not report anything that can not be verified
- Unique count: It is common that beneficiary/stakeholders attend multiple activities such as training classes, events etc. In such a situation, it is better to report unique and total counts both.
Automate your impact data aggregation for continuous feedback
You are here.
While 95% of social sector organizations use Excel or Google docs as data management tools along with other third party reporting dashboards such as Google Data Studio or Tableau, this strategy is likely to create a broken process. First you must design a data aggregation strategy. Tools like Excel or Google docs are easier to use but they create many data management challenges such as:
- Data loss when the file manager leaves
- Formula update every-time you bring new data
- Tracking original sources of data
- Continuously changing impact metrics and data strategy
To avoid year-end data scrambling, you want to be here.
Average social sector organizations have anywhere from 1 to 50+ data sources internally. Each of them may have different types of stakeholders with varying requirements of reporting, such as:
- Report to funders on regular frequency like monthly, quarterly, semi-annual or annual period
- Narrative oriented report with emphasis on the robust storytelling
- Individual and personalized reports to the large donors
- Grantee or investment effectiveness reports
- Dashboard for internal operations and other KPIs
Management should strive to build a robust data warehousing-like application, to improve high-quality impact data evidence and reporting. Most Excel or Google docs-like systems will lose data integrity over time. Instead, a system that can provide high-quality reporting is needed.
So how do we do that?
- Build a simple data warehouse-like system that simplifies data aggregation from most common data sources that are important for impact reporting
- Ideally, that system should allow you to build formulas at the database level, so that user do not have to update them every-time
- Provide simple data analytics to find answers to most frequent output, outcome and unique data reporting with the ability to visualize results based on reporting type
Define an impact data roadmap
- There are many objectives of your social impact journey. Depending on our internal management goals and funder requirements, you must define priorities. Based on each priority, now you will continue to add metrics to your impact strategy (theory of change, logic model or Impact Management Project dimensions). While it is not necessary to do all, depending on your internal goals you will need to develop a systematic plan to scale your program. Your plan should include:
- Stakeholder voice - immediate feedback
- Stakeholder voice - qualitative feedback
- Performance tracking
- Outcome tracking
- Stakeholder voice - impact management approach
3. Start with stakeholder voice
It starts with a well-defined survey and a better engagement process
A well-designed stakeholder survey is the foundation to start building an impact management journey. The key to successful stakeholder listening requires:
- Well-designed surveys with more precise questions and a mix of open-ended and closed-ended questions
- Various dimensions of feedback specific to product or services provided by the organization
- Cost-effective surveys with a clear collection method (door to door, email, SMS, call center), etc.
- A decent size of a sample to survey (typically around 5% to 15% depending on the size of beneficiary or stakeholder pool)
The following customer impact report helps you understand the methodology, questions, and results for a publicly-traded company & community development institution.
4. Integrate qualitative and quantitative analysis
While the first generation of the survey can be quantitative, a well-designed approach may include qualitative data. An excellent example of a qualitative approach is the interview. Specific well-designed open-ended question will offer a real insight into what stakeholder thinks. A closed-ended question often fails to capture the stakeholder sentiment as they focus on what the organization wants to learn and not what the stakeholder wants to say.
There are many different ways to learn what the stakeholder says. A few examples are:
- A poll question followed by opened-ended and well-defined follow-up questions that can be captured through social media, interviews, or survey questions.
- Analyzing emotions and tones in what people write online, like social media feeds, impact reports, or reviews. By processing qualitative text analysis through Artificial Intelligence can give you tremendous insights into what your stakeholders are saying.
A well-designed approach can capture real sentiments at the individual stakeholder and group level. The following example gives the overall summary of critical emotions:
- Predict whether they are happy, sad, confident, and more
- Key challenges facing product and services
- What are the other issues they would like for your organization to address in the future
By linking qualitative response with quantitative or closed-ended question results as seen in the scorecard below, you can quickly understand causality or correlation between crucial indicators.
Simple emotional analysis & Integrated Scorecard
While qualitative sentiment analysis is valuable in understanding beneficiary voice. The real value comes after analyzing and correlating qualitative and quantitative analysis. Following example impact scorecard gives an in-depth understanding of how you can analyze and communicate results effectively:
5. Start with performance management
The mission-driven organizations should clearly define critical indicators that are key to measuring the overall success of the program. These indicators may be activity and/or output-oriented, but they must refer to the overall progress to evaluate internal performance.
Compare cohort results over a period of time
Analysis can often be derived from existing results tracking systems such as educational scores of students. Key to such analysis is to conduct the study under similar conditions, with a non-biased approach.
In this example, we are able to gather learnings like:
- The average score of participant grew from 4 to 6
- 35% of the cohort had a score greater than 6 at the end of the program
- 96% of participants have seen the positive change in the positive average score
Compare individual stakeholder results over a period of time
Target audience: Impact Analyst, Evaluation Team
In this specific case, it may be necessary to look at the patterns of improvement between multiple stakeholders, to see if a particular approach is working better for individual students.
.png?width=894&name=monthly-education-score%20(1).png)
Compare program results with key constraints
Target Audience: Impact Analyst, Evaluation Team
Funders may be looking to learn more about social, economic, and financial results. Before starting, funders must define a clear hypothesis of the improvement that they are seeking, such as:
- Total monthly salary improvement over a period of time
- Overall monthly salary improvement over a period by Gender
Often this requires comparing multiple metrics and filtering/grouping by specific criteria such as gender, district or education level. If they do not see necessary improvements, funders can initiate an action or strategic plan using the impact scorecard.
6. Build a good outcome tracking strategy
Outcome tracking should be designed based on a combination of subjective analysis and evaluative approach.The fundamental criteria here is to review the holistic improvement or observe a specific limitation over a period of time. This analysis not only confirms the overall breadth of improvement, but also points out the deficiencies in the program approach.
7. Impact management project stakeholder
The Impact Management Project has made significant progress by bringing different impact practitioners together to agree on a standard definition of impact, and the dimensions of performance that matter for impact measurement, management and reporting across the value chain. While this approach is likely to be well-adapted in the impact investment industry, we believe that it could and should be adopted by the philanthropic organizations. On the other hand, businesses and ESG investments are already paying attention to this approach.
How to design an IMP-based stakeholder survey
Enterprise or mission-driven organizations might want to start by understanding who is their stakeholder by collecting data on their demographics. Often time, this information is already available in their own internal systems such as student enrollment, loan management systems, etc. Generally, demographic information should be collected based on appropriate data collection for each of the stakeholders (WHO) and not based on a selected survey.
Enterprises need further data on which impacts are most important to the stakeholder. They also need to assess the degree of change (How Much) that impact has occurred for each of the stakeholders.
| What | |
| Impact Category | Commentary |
| Outcome in period | The outcome experienced by the<stakeholder>when engaging with the enterprise. |
| Threshold for positive outcome | The level of outcome that the<stakeholder>considers to be positive/good enough. Anything below this level is considered negative/not good enough. |
| Capital type | The type of capital that the outcome relates to. |
| SDG | The Sustainable Development Goal that the outcome relates to, along with the specific target and indicator. An outcome may relate to more than one SDG. |
| SDG target and indicator | |
| Importance of<outcome>to stakeholder | Stakeholder’s view of whether the outcome they experience is important. Where possible, the people experiencing the outcome provides this data, e.g. through direct surveying, although third party research may also be included. For the environment, scientific research provides this view, such as the work of the Stockholm Resilience Centre. |
8. Social return on investments
Social Return on Investment (SROI) defined by Social Value International is a pioneering approach to engage in stakeholder-based social value accountability. Social return on investment (SROI) is a principles-based method for calculating a social return valuation. This method allows to evaluate the impact on stakeholders and create a better performance measurement for investments.
SROI accounts for the stakeholders view of impact, similar to Impact Management Project (IMP). However, unlike IMP, it requires financial 'proxy' values on all those impacts identified by stakeholders which do not typically have market values. The challenge, however, is that the 'proxy' is often context-sensitive and finding standard proxies may not be easy and may require additional cost. On the other hand, SROI does give a great amount of guidance to an investor if they are able to define high-quality proxies considering the stakeholders views.
It is important to understand that SROI is one of the mechanisms to demonstrate impact but it should be compared and valued against other alternatives described in this article.
Conclusion
There are many ways to demonstrate social impact of your mission. Depending on the maturity of your organization you can start your journey and build a step by step strategy to learn and demonstrate impact. We demonstrated following commonly used techniques that will surely put you on path to raise impact capital from your mission.
- Define Impact Strategy
- Programmatic Reporting: Financial, Demographics, Operation, Donors
- Stakeholder voice
- Performance Improvement
- Outcome Improvement
- Impact Management Project based Stakeholder Analysis
- Social Return On Investment (SROI)
- Aggregating results from partners
Learn More:
