While the concept of Monitoring & Evaluation has been around for decades, use of Monitoring & Evaluation software has grown ever since US Congress passed a law requiring International Development Programs to set aside M&E funding for every grant allocation. While the intent of this was to bring accountability & regulatory, the real action was focused on regulatory, so that all the grant-making authority could manage seamless data aggregation from grantees or agencies. This gave rise to M&E software funded by USAID,
Monitoring & Evaluation software provides an important foundation which is necessary for aggregating results from different partners or agencies. While many advance platforms have built-in data collection for disaggregated results for their results framework, the onus is on partner/agency/grantee to provide individual results in form of data tables. This data table in turn provides calculation capability so that funder can see individual & aggregated results. While this works very well from the funder perspective as they get complete results from the field, this approach is fundamentally lacking an important lesson of accountability from impact investment community.
One the important framework for results collection
The rise in impact investment has given a fresh understanding of what impact is? Impact Management project is one the leading project that provides a deeper insight into how impact should be managed instead of evaluated. True impact requires that we rethink, how we manage impact effectively. The goal of M&E should not be about collecting results from downstream partners but to make them accountable for their data and impact. They must have a right size process that allows them to collect feedback in a systematic way. While it is easier for the downstream partner to collect data on paper, excel or homegrown systems and report using M&E software interface, often missing in action is a system that allows them to manage the beneficiary outcome. These outcomes are often collected in the survey tools like google spreadsheet or SurveyMonkey, offline, voice/SMS, Salesforce, qualitative systems like NVivo or partner records such as email or Word/XLS. Regardless of the source of the collection, it is first and foremost, the downstream partner must have good systems to collect both qualitative and quantitative evidence. More importantly, these systems should allow them to collect actionable results, which allows a better feedback system. The role of downstream partner (NGO, Agency, Grantees) should be more like a startup who continues to qualify or negate hypothesis of their impact intervention.
It is the downstream partner who should be collecting data, aggregating them by funder requirements and managing own performance requirements to meet theory of change (TOC). We firmly believe that downstream partner should have better systems so that they can
- Manage disaggregated data effectively for actionable results. In fact, M & E software should not be providing a capability for grantees to enter disaggregates or data table, as it defeats a purpose of data accountability. If grantee, do not manage those data themselves, they are not using data to achieve their TOC. Funders only have an incentive to collect those results so that they can provide their internal reports. However, these results are not actionable enough.
- Track any key results (both qualitative and quantitative) in a single data store to reduce island of data into different systems
- Downstream partners (grantees, agencies etc) should easily be able to see results based on real-time calculation of outcomes & evidence so that they can improve their results continuously
In the next article, we will describe a future of new M&E Software or impact management solution to meet these goals. By seamlessly integrating multiple systems into a single platform and reducing time spent on consolidating data, thereby allowing the organization to spend more time analyzing data, build impactful reporting engine that can rapidly integrate qualitative and quantitative outcome and take action.
Moving from Past M & E Software System
We looked at how M&E software systems are still failing to achieve key data accountability goals. In this article, we offer an innovative approach that focuses on key principle. A simple, flexible and comprehensive data management solution (similar to data warehouse found in corporate IT world, but non-existent in the social sector IT solutions ). Most users in social sector IT are resource crunched. While this parallel analogy helps understand business problem, there are two main differences
a) Most users, including those in IT, are looking for lot simpler systems
b) Data sources are quite different from corporate IT (such as ERP, CRM, Order Management, HRM, etc.)
The key requirements for these systems are:
- Track results in a single data store to reduce island of data into different systems
- Real-time calculation of outcomes identified by an organization
- Beneficiary unique reporting
- Pre-defined templates for each partner and funders to generate quick reports
- Scorecard or SROI for impact evaluation
- Cloud-based integration & storage (such as Zapier) for collecting results from unstructured or qualitative data management software
Types of Monitoring and Evaluation Software Systems
Based on these requirements, there are two good alternative options.
Option 1: Cloud Platforms
During last decade there are several leading robust cloud based platform have emerged, such as Salesforce and Microsoft Azure (these are two enterprise IT solution, but there are many more).
Many organizations are already using Salesforce for much of its CRM processes and see an opportunity to expand the use of Salesforce perhaps to include other needs including monitoring, evaluation, and learning system. However, an average organization can take months customizations and large IT budget to achieve a final system. While many organizations are moving towards these platforms, many are finding difficult to use efficiently beyond their core functionalities such as CRM, fundraising tool, etc. These platforms require significant customizations. It's a non-issue in a normal corporate IT as they may have the capacity to fund these skills internally, they can build application internally or externally as they have higher technology learning curve. Social sector organizations, on the other hand, are dependent on grants based model, don't seem to have a steady investment for highly sought IT skills (unless large non-profit). As a result, they tend to struggle to come up with quality requirements. Often they have to rely on IT resources which are usually not coming from social sector and probably not familiar with M&E software.
Even after building a system, Salesforce/Azure platform is questionable from an average IT user in Social sector organization. While it is easier to store quantitative data, it is challenging to integrate dynamic narrative from unstructured data sources, qualitative results to build impactful reports.
By seamlessly integrating multiple systems into a single platform and reducing time spent on consolidating data, thereby allowing organizations to spend more time analyzing data, build impactful reporting engine that can rapidly integrate qualitative and quantitative outcome and take action. While this practice is very common in corporate IT world, it is in not common social sector IT.
Option 2: System for Progress Data & Coexist with current system
The concept of a data warehouse is very common in corporate IT. This concept is virtually nonexistent in the social sector IT except with well funded nonprofits and foundations. While we certainly do not preach the same in the social sector IT, SoPact Impact CloudTM is a virtual data warehouse that simplifies data aggregation from any sources of data.
The modern impact management system should continue to coexists with any internal systems, not create another data island and provide a comprehensive and flexible way of managing evolving impact & progress data management.
Another challenge is that social sector often collects data from disconnected data sources such as qualitative systems, unstructured data sources such as excel/word, shared cloud drive and structured data stores.
A properly designed system not only provides hosting for all progress data from partners, beneficiaries or any other sources of data but also provides a simple way of querying results to meet different funding and program cycles. This system should provide historic results in a trackable database and not in islands of excels. The system needs to be designed based on modern cloud-based integration principles and online services such as Zapier, OpenFn, Twilio, Workato, etc as data sources in the social sector are located in disconnected sources. Once these data is aggregated, program manager should be able to quickly build impactful reports with simple drag & drop tools.
Learn More: Monitoring and Evaluation
Written by Unmesh Sheth
Unmesh is the founder of the SoPact. SoPact is a personal vision that grew from 30 years of experience in technology, management, and the social sector.