Key Players: Program Director, Learning & Evaluation Director, Program Analysts, Evaluator and IT Director & Team
How would a foundation know that their funded intervention is generating an intended outcome? While many large foundations may have their internal metrics, these are often not consistent with their mission. Also, most of these metrics are output oriented and not outcome oriented. As as a result, foundations cannot understand which intervention is critical for improving the outcome. If they collected the right metrics that allow them to know both, they would be in a much better position to understand how particular programs can be made effectively in certain communities.
Also, the most foundation takes a long time to finalize their core, program or partner specific metrics.
Granularity and context of impact are often missing in metrics shared with grantees or program agencies. While this granularity is defined most, still use monolithic metrics by a program, making it not relevant to each impact organization. In other words, foundations typically use a survey oriented outcome data collection approach. Thus many grantees or program agencies balk at a process of filling out a report which doesn't have metrics specific to their program context. Finally, many organization take months or over a year in some cases to finalize relevant key metrics.
Challenges with grant management based outcome measurement
While most midsize family foundations are just happy to receive reports on a defined frequency, mid-size foundations may vary. Impact measurement is gradually moving up on their radar, their metrics and indicator selection can be all over the place. In many cases, they frequently ask grantees to come up with core metrics, and their evaluation staff or grantee office reviews findings. Most organizations feel that they do not have time & resources to monitor metrics across their programs consistently.
Mid Size Foundations → Might go in other strategy
While most midsize family foundations are just happy to receive reports on a defined frequency, mid-size foundations may vary. Impact measurement is gradually moving up on their radar, their metrics and indicator selection can be all over the place. In many cases, they frequently ask grantees to come up with core metrics, and their evaluation staff or grantee office simply reviews findings. Most organizations feel that they do not have and time & resources to build metrics that can be consistently monitored across their programs.
Many time the mid-size foundation has one or two staff devoted to grant, impact and data management. They often don’t have any IT resources in-house. Hence, they often rely on cloud-based third-party applications. While they may have some grant management solution, they are still collecting impact through annual reports. For these foundation collecting data through online & managing data is perhaps a great first step.
Aggregate Data & Report
Most grantors collect data as a to meet internal board mandated requirements. Often these results come in the form of annual PDF and/or XLS reports.
As a result, their collection doesn't reflect the long-term impact created by grantees and program agencies or the broader program results. In many cases, large foundations have staff assigned to nudge large numbers of grantees to provide results, just to put the results on foundations local drive or grant management software. While some of the foundations are starting to employ the impact measurement module, their use is primarily limited to documentation or to publish results for a goodwill generation. Very few have figured out a way to receive program outcome data so that they can understand their key interventions, and best practices to drive impact through others.
While visualization and story building is becoming synonymous with measuring impact, the focus on communication is overwhelming and not doesn't reflect the accurate understanding of impact evidence. The initial push-back against measurement comes with a lack of knowledge of how the analysis should be done!
A major roadblock often comes from a simple catchphrase "Social Impact Measurement Is expensive!" Our articles series in NextBillion.Net explain that advancement in technology in recent years have often brought down the cost of collecting data at the beneficiary program level at an affordable level. For funders like Foundations, Impact Cloud represents a comprehensive solution at a very affordable cost compared to an internal custom software solution.
Foundations can design hierarchical programs that allow collecting common metrics at the top level (e.g., Education common metrics), creating subprogram (e.g., country-based such as Africa, India, US or sub-domain based such as Primary Education, Secondary Education, College Readiness, and Mentoring program). Finally, the third level can have the value unique to organizations with similar outcome and impact.
This approach is very different from most grant management platform based approaches such as EffortToOutcome (ETO), Microedge and Fluxx. In those programs, foundations are forced to create a row form specific to each organization to collect actual impact which is quite time-consuming, and data cannot be adequately aggregated as everyone has a different metrics. If an organization uses the same metrics across all the grantee, they do not have an incentive to provide results as often grantee perceives data collection not related to their impact.
SoPact provides a broad range of data collection for different types of grantees data capacity.
- For grantee with limited internet connectivity, offline data collection in step by step manner
- Batch data import for historical data migration
- A third party of operator based data where data authenticity or accuracy is often challenging
- Google form or Survey Monkey like data collection for grantees who often do not want to have any learning curve
- Batch online data entry for program staff
- Partner (Grantee, investees, and project agencies) should be reported step by step, interpret and learn the intent of each metrics carefully and provide data results with ease & standardized data collection
- The best part of data entry process is that each form collects data by specific frequency (monthly, quarterly, semi-annual, and yearly).
- Each form metrics can be represented in different languages and converted to different reporting language.
Each metrics contains proper context such as usage guideline, example answer, help text, an opportunity to provide flexible supporting documents such as picture, video, document, and story. Each metrics automatically populates proper data collection and validates results. Foundation can decide to combine both qualitative and quantitative indicators to understand causality and correlation between multiple indicators.
Finally, program analyst team can schedule all the data collection based on reporting requirements and track data collection progress from each partner including % completion, development, stated, finalization and reject results.
Also, Impact Cloud provides many important features. While not all features may not be critical to all your internal needs, please review details here:
- Impact Framework
- Project Portfolio
- Investment Portfolio
- Impact Insight
- Feedback Loop & Messaging
- Impact Reports