If you are a nonprofit, international development organization, or social enterprise, you must be collecting the data to have continuous learning and engaging donors. You also have to report your program outcomes to your stakeholders. Is your Monitoring and Evaluation system working for you to provide above needs to build compelling reports that align with funder's agreement?Today we will talk about five challenges of current monitoring and evaluation software and how to improve outcomes using continuous learning and improvement based approach.
While many nonprofits and social enterprises may have Monitoring and Evaluation software, often they are designed around a results-based framework. Results based framework is often tightly coupled between different programs, focuses on project management with a strong emphasis on collecting activity and output results. These systems are often too simplistic or too complex. Either they focus on simple data collection, visualization, and documentation. Or some are sophisticated but overly complicated, with lengthy implementation, difficult to improve once implemented, and takes a long time to implement.
Most monitoring and evaluation systems fail to provide scalable solutions to aggregate results regularly. Those that do may require a significant amount of customization or manual data aggregation. So without further ado, let's dive into the five worst mistakes to avoid when choosing monitoring and evaluation systems.
Start with continuous learning and improvement to improve mission
Most organization’s monitoring and evaluation systems often designed on simple data collection. Often these data are activities performed by the program teams. So, year-end reporting to funder would be “1.5 Million Meal Served”, “250000 members served”, “9.5 Million Beneficiaries Impacted”. While these numbers look impressive, how do we know if these numbers validate an outcome and prove the mission of an organization?
As social value principle number four says, “only include what is material – determine what information and evidence must be included in the accounts to give a true and fair picture, such that stakeholders can draw reasonable conclusions about the impact”
Often these numbers may look impressive, it may be impossible to assess the true impact of a program. This is why the principle five of Social Value International says “Do not over-claim – Only claim the value that activities are responsible for creating”.
Rather than focusing on activities, a key focus should be on getting continuous feedback from stakeholders. If you are working on mother and child mortality and looking to improve healthcare data collection through mobile technology, first create a feedback system with the healthcare field workers to understand adoption and satisfaction. Only after iterating and improving, focus on improving the health outcomes of remote patients. And remember activities and output numbers are not meaningful if you plan to scale the mission.
Second, emphasis should be on understanding what changes – Articulate how change is created and evaluate this through evidence gathered, recognizing positive and negative changes as well as those that are intended and unintended (Social Value Principle 2)
A rigid data management system can kill user adoption
There are a plethora of monitoring and evaluation systems to choose from and some of them are really great and have been around for a long time. While choosing a system an organization should always start with analyzing their own needs in the short term as well as in the long term. Factors such as time for implementation and long term costs must be considered. Many Monitoring and Evaluation software started in early 2000. Many of them began with a J2EE or Client/Server architecture and later were rebranded as Cloud-based software. Often, these platforms take a lot longer than most realize, increasing implementation risk or longer deployment time and cost run-up. For that matter, Salesforce, which is the most popular cloud-based platform, can end up taking very long. We know of many organizations starting on this path taking over a year. Even if they managed to go live, our review found that their platform was giving them limited analytics and reporting.
Are you familiar with QuickBooks? A popular US-based accounting application? This is how most configuration based application looks like! They are easy to use, flexible, and yet comprehensive.
Just as QuickBooks, new Monitoring and Evaluation systems should allow organizations to log in and let the online application guide them on the best way to implement their impact strategy, metrics, and survey, and deploy in a short time. This approach puts program management or evaluation management staff in the driving seat, allowing them to configure changes over a long time adapting to changing metrics or reporting requirements.
Modern cloud-based must be best of the breed
Often an organization looking for Monitoring and Evaluation software. Often these teams do not have experience in defining monitoring and evaluation software requirements. Given that they hire a consultant with prior knowledge of monitoring and evaluation process. Often the external consultant starts with a monitoring and evaluation checklist and works with the organization to refine the requirements. The challenge here is that often consultants often may not have extensive experience with implementation.
There are two challenges with this approach., They often come up with requirements such as project management, complex workflow, log frame-based reporting for activities. Unfortunately, these kinds of requirements may not even be needed. And even if they are needed, often current cloud-based software has now evolved towards the best of breed approach. For example, there is a high probability of user adoption if they use online project management software such as Trello, Asana, Basecamp or monday.com compared to project management integrated within the monitoring & evaluation software. Best of breed based approach often increases user adoption, cost and implementation time. All you have to do is to make sure that the vendor can provide the right integration between two applications.
Many complex monitoring and evaluation systems resemble old IT-based applications like ERP systems. The most modern system often focuses on building the best user experience with a complete lifecycle of applications. If you need other features, just ensure that the application has a smooth integration approach.
Design a system that uses the theory of change throughout the lifecycle
Theory of change or Log Frame is a great starting point for defining organizations, short term, midterm, and long term outcomes. However, these tools are not just useful for defining the right approach, but these approach needs to put in the action through a better data strategy. Often time, outcome and performance monitoring requires a well-designed strategy.
- How are you going to design a stakeholder survey?
- How are you going to define baseline, mid-line and exit line surveys?
- What is the nature of the survey?
- How do you plan to integrate qualitative questions and analyze what stakeholder has to say?
- What kind of continuous learning and improvement needed?
- What is a source of data and how do you plan to collect data?
Most organizations often start with a waterfall-based approach. The waterfall model is a breakdown of project activities into linear sequential phases, where each phase depends on the deliverables of the previous one and corresponds to a specialization of tasks. This approach was the most common a couple of decades ago and many monitoring and evaluation experts still follow this approach.
The better approach is to design a blueprint or roadmap of improving impact management through step by step iterative or agile approach. In this, you will build a lean data approach starting with stakeholder-based data collection. Gradually look to improve your framework to prove the results of the programs. Once a model has been proven with the change that you are looking for and the stakeholder is fully agreed with your approach, your system can ultimately focus on optimization necessary to scale your mission.
Streamline reporting for different donors or funders
Many organizations often receive funds from different sources - grant makers, individual donors, investors, and so on. Even different grant makers may have different reporting requirements. Unfortunately, many organizations have different program data management systems making it difficult to produce results. Often these program systems such as case management, an effort to outcome and reporting may have built-in reporting but limited to operational reporting. This makes it difficult to align current systems with a theory of change or funder reporting requirements.
In many cases, field managers collect data on individual worksheets (excel or google), making it difficult to streamline consistent reporting for each funder. In fact, most mission-driven organizations often fail to look at the value of impact reporting to funders.
A well-designed system can provide role-based access to individual program teams to manage their data and reporting with consistent reporting with high integrity and quality. This system should not be manual and should not add a burden to the reporting team. When you report results to funders with pre-defined KPIs and reporting period, funders will build trust, and become long term contributors reducing dependence on marketing-driven fundraising.
That wraps up tips for the day
Here at sopact, we understand that organizations just like your’s face challenges with impact management on a daily basis. So we’ve developed a platform that streamlines the process of impact management.
“Impact Cloud” simplifies the different forms of data collection and aggregation through smart field mapping, realtime and mobile integration and database level formula. This simplifies your data collection, data tracking and reporting through a simple system.
With this approach, you will -
- save months worth of reporting time
- focus on high-quality data collection and over number-crunching
- align your reporting to each funder’s metrics and reporting frequency
If data is the lifeblood of your organization, you should select your data collection tools appropriately.