If you are a nonprofit, international development organization, or social enterprise, you must be collecting the data to have continuous learning and engaging donors. You also have to report your program outcomes to your stakeholders. Is your Monitoring and Evaluation system working to provide the above needs to build compelling reports that align with the funder's agreement?Today, we will discuss the five challenges of current monitoring and evaluation software and how to improve outcomes using continuous learning and improvement-based approaches.
While many nonprofits and social enterprises may have Monitoring and Evaluation software, they are often designed around a results-based framework. The results-based framework is often tightly coupled between different programs and focuses on project management, emphasizing collecting activity and output results. These systems are often too simplistic or too complex. Either they focus on simple data collection, visualization, and documentation. Or some are sophisticated but overly complicated, with lengthy implementation, difficult to improve once implemented, and takes a long time to implement.
Most monitoring and evaluation systems fail to provide scalable solutions to aggregate results regularly. Those that do may require a significant amount of customization or manual data aggregation. So without further ado, let's dive into the five worst mistakes to avoid when choosing monitoring and evaluation systems.
Start with continuous learning and improvement.
Most organization’s monitoring and evaluation systems are often designed on simple data collection. Often these data are activities performed by the program teams. So, year-end reporting to funder would be “1.5 Million Meal Served”, “250000 members served”, “9.5 Million Beneficiaries Impacted”. While these numbers look impressive, how do we know if these numbers validate an outcome and prove an organization's mission?
As social value principle number four says, “only include what is material – determine what information and evidence must be included in the accounts to give a true and fair picture, such that stakeholders can draw reasonable conclusions about the impact.”
Often these numbers may look impressive; it may be impossible to assess the true impact of a program. This is why principle five of Social Value International says, “Do not over-claim – Only claim the value that activities are responsible for creating.”
Rather than focusing on activities, a key focus should be on getting continuous feedback from stakeholders. If you are working on mother and child mortality and looking to improve healthcare data collection through mobile technology, first create a feedback system with the healthcare field workers to understand adoption and satisfaction. Only after iterating and improving, focus on improving the health outcomes of remote patients. And remember, activities and output numbers are not meaningful if you plan to scale the mission.
Second, emphasis should be on understanding what changes – Articulate how change is created and evaluate this through evidence gathered, recognizing positive and negative changes as well as those that are intended and unintended (Social Value Principle 2)
A rigid data management system can kill user adoption.
There are many monitoring and evaluation systems to choose from, and some of them are really great and have been around for a long time. While choosing a system, an organization should always start analyzing its own needs in the short and long terms. Factors such as time for implementation and long-term costs must be considered. Many Monitoring and Evaluation software started in early 2000. Many of them began with a J2EE or Client/Server architecture and were rebranded as Cloud-based software. Often, these platforms take a lot longer than most realize, increasing implementation risk or longer deployment time and cost run-up. For that matter, Salesforce, the most popular cloud-based platform, can take very long. We know of many organizations starting on this path, taking over a year. Even if they managed to go live, our review found that their platform gave them limited analytics and reporting.
Are you familiar with QuickBooks? A popular US-based accounting application? This is how most configuration based application looks like! They are easy to use, flexible, and yet comprehensive.
Just as QuickBooks, new Monitoring and Evaluation systems should allow organizations to log in and let the online application guide them on the best way to implement their impact strategy, metrics, and survey and deploy in a short time. This approach puts program management or evaluation management staff in the driving seat, allowing them to configure changes over a long time, adapting to changing metrics or reporting requirements.
Modern cloud-based must be the best of the breed.
Often an organization is looking for Monitoring and Evaluation software. Often these teams do not have experience in defining monitoring and evaluation software requirements, given that they hire a consultant with prior knowledge of the monitoring and evaluation process. Often the external consultant starts with a monitoring and evaluation checklist and works with the organization to refine the requirements. The challenge here is that often consultants may not have extensive experience with implementation.
There are two challenges with this approach., They often develop requirements such as project management, complex workflow, log frame-based reporting for activities. Unfortunately, these kinds of requirements may not even be needed. And even if they are needed, often current cloud-based software has now evolved towards the best-of-breed approach. For example, there is a high probability of user adoption if they use online project management software such as Trello, Asana, Basecamp, or monday.com compared to project management integrated within the monitoring & evaluation software. The best breed-based approach often increases user adoption, cost, and implementation time. All you have to do is make sure that the vendor can provide the right integration between the two applications.
Many complex monitoring and evaluation systems resemble old IT-based applications like ERP systems. The most modern system often focuses on building the best user experience with a complete lifecycle of applications. If you need other features, ensure that the application has a smooth integration approach.
Design a system that uses the theory of change throughout the lifecycle
Theory of change or Log Frame is a great starting point for defining organizations, short term, midterm, and long-term outcomes. However, these tools are useful for defining the right approach, but these approaches need to put into action through a better data strategy. Often time, outcome and performance monitoring requires a well-designed strategy.
- How are you going to design a stakeholder survey?
- How are you going to define baseline, mid-line, and exit line surveys?
- What is the nature of the survey?
- How do you plan to integrate qualitative questions and analyze what stakeholder has to say?
- What kind of continuous learning and improvement needed?
- What is a source of data, and how do you plan to collect data?
Most organizations often start with a waterfall-based approach. The waterfall model is a breakdown of project activities into linear sequential phases. Each phase depends on the deliverables of the previous one and corresponds to a specialization of tasks. This approach was the most common a couple of decades ago, and many monitoring and evaluation experts still follow this approach.
The better approach is to design a blueprint or roadmap of improving impact management through step by step iterative or agile approach. In this, you will build a lean data approach starting with stakeholder-based data collection. Gradually look to improve your framework to prove the results of the programs. Once a model has been proven with the change you are looking for, and the stakeholder fully agrees with your approach, your system can ultimately focus on the optimization necessary to scale your mission.
Streamline reporting for different donors or funders
Many organizations often receive funds from different sources - grantmakers, individual donors, investors, etc. Even different grantmakers may have different reporting requirements. Unfortunately, many organizations have different program data management systems making it difficult to produce results. These program systems, such as case management, an effort to an outcome, and reporting, may have built-in reporting but are limited to operational reporting. This makes it difficult to align current systems with a theory of change or funder reporting requirements.
In many cases, field managers collect data on individual worksheets (excel or google), making it difficult to streamline consistent reporting for each funder. In fact, most mission-driven organizations often fail to look at the value of impact reporting to funders.
A well-designed system can provide role-based access to individual program teams to manage their data and reporting with consistent reporting with high integrity and quality. This system should not be manual and should not add a burden to the reporting team. When you report results to funders with pre-defined KPIs and reporting period, funders will build trust and become long-term contributors reducing dependence on marketing-driven fundraising.
That wraps up tips for the day.
Here at sopact, we understand that organizations just like yours face challenges with impact management daily. So we’ve developed a platform that streamlines the process of impact management.
“Impact Cloud” simplifies the different forms of data collection and aggregation through smart field mapping, real-time and mobile integration, and database-level formulas. This simplifies your data collection, data tracking, and reporting through a simple system.
With this approach, you will -
- save months worth of reporting time
- focus on high-quality data collection and over number-crunching
- align your reporting to each funder’s metrics and reporting frequency
If data is the lifeblood of your organization, you should select your data collection tools appropriately.