5 Steps: How to Design Social Impact Evaluation Plan?
May 6, 2019
Impact evaluation is the process where we study the results of social interventions. We ascertain the value of the overall initiative and use the findings as learnings for future endeavors. This is a critical process for both the investors to understand the rationale behind continuing the program and for impact makers to see if their efforts are bringing positive change. This article underlines the steps you need to take to define an effective impact evaluation plan.
Program theory is the step where you document the assumptions and logical arguments that define the rationale behind your program. If you would like to take a more sophisticated approach we recommend to go a level higher and do as below:
Theory of change is the program theory where you not only describe the assumptions and logical arguments in favor of your initiative but also lay down all the possible scenarios that can result from your impact actions. This gives a clearer picture and helps you understand the evaluation reports from a wider perspective.
Here is all you need to know to define Theory of Change for your program.
In its essence counterfactual analysis is a “with versus without” analysis. We study the impact of a program by comparing the results from a control group where no artificial factors were stimulated. You need to set up this control group in the early stage so that the evaluation can be done parallelly and results are comparable. In some cases, this can be avoided where there is no other factor that can bring any observed change in outcomes (e.g. reductions in time spent fetching water after the installation of water pumps).
The presence of selection bias can skew the results of evaluation hence, it is important that we take whatever steps possible to eliminate all bias. Below are the preconditions to identify the extent of selection bias and the remedial step that can be taken to eradicate it:
We cannot address the problem of selection bias if none of the above are possible. Thus, we are left to rely on program theory and triangulation to build an argument. Thus, setting up a Theory of Change (TOC) Model helps make plausible association easy.
A baseline survey is performed when the project is being initiated i.e. at the beginning of the project but implementation. It helps prioritize between different objectives of an initiative and works as a benchmark to identify the success or failure of it.
If you are reading an article on Impact Evaluation procedure chances are that you are in the end of the program phase. Thus, if the baseline survey is missed in the beginning you cannot go back to collect that data. Here are a few things we can do:
Triangulation means using different types of samples and methods of data collection. Thus, we can ensure the validity of results by comparison. This step becomes all the more necessary when we cannot eliminate selection bias or establish an authentic baseline. Triangulation helps build confidence in findings and fills in gaps in statistical studies. Be sure to allocate part of your budget and time to this step.
The purpose of the evaluation is not just to measure but evaluate results. In this step, we need to weigh in the qualitative data alongside the numbers gathered to make a deeper study on the impact actions. The feedback from field data collectors comes handy here. They can give perspective on the situation, status and people lives as well the authenticity of the experiment. The second type of qualitative data would be the inferences are drawn and cause-effect relationships established at the beginning of the experiment. Do they still go in-line with the numbers? If not, where did you miss?
These are some of the steps you can take to effectively conduct an impact evaluation. Be sure to check out our complete guide to Actionable Impact Management to further understand the process of social impact monitoring, evaluation and assessment.
Topics: monitoring & evaluation
From starting my own blog to managing accounts in an advertising company I have worked in the field of branding and communication from all directions. Now I lead a media agency and help businesses discover their potential with digital marketing.