Evaluation Models Depend on Organizational Context
Determining an evaluation method requires first defining the unique objectives of the organization or specific program. In general, those objectives are determined by asking: What is the impact being sought? What is the purpose of the evaluation?
In addition to those objectives, other variables include, the context of the target beneficiary group and the organizational resources available (money, skills, tools, etc.). Of course, focus should be given to achieving the overarching objectives.
The following, while not an exhaustive list, are some examples of the main evaluation approaches which accomplish unique organizational objectives.
A formative evaluation will most often be conducted before a program begins to examine both feasibility and determine its relevance to the strategic objectives of the overall organization. This can also occur during program implementation, especially if there is a need to modify the program, at which point the formative evaluation can be used to assess feasibility of a new design.
Part of the importance of a formative evaluation can improve a program’s probability of success because it encourages practitioners to confirm viability and detect potential problem areas at the outset, while also promoting accountability during implementation.
A process evaluation is carried out during the implementation phase of a program. As the name suggests, it focuses on processes being carried out -- inputs, activities, outputs, etc. It identifies any issues with the efficiency of implementation.
For example, it can establish whether targets were not met because of lack of human resources (skills) or appropriate tools, or unforeseen contextual obstacles (e.g. beneficiaries lacked time to engage with program) which ultimately affected program outcomes.
If there don’t seem to be process-related issues, the evaluation can help illuminate issues with the change model itself, encouraging a needed rethinking of how to affect change for the target group.
The use of periodic assessments during implementation is one of the most important process evaluation components. This allows organizations to re-design if needed during execution to increase reach, re-allocate resources, etc.
An outcome evaluation aims to determine whether overall program objectives have been met. In that process, practitioners also identify what might have spurred or limited those changes. Finally, it helps shed light on unexpected changes in the target beneficiary population at the end of the intervention or at the point of evaluation.
Given that scope, an outcome evaluation generally looks at a program’s results over longer period of time (although this also depends on how quickly or slowly the change is expected to occur).
It can also help pinpoint which areas of a program were more or less effective than others. Most importantly, with an outcome evaluation an organization determines whether there was a change in the lives of beneficiaries.
For this reason, it can be important to use qualitative measures and participatory methods to extract from beneficiaries their perception of any observed changes.
An impact evaluation gets to the heart of a program’s true effectiveness by determining attribution, or to what extent the changes observed (outcomes) can be causally connected to the activities carried out during the program period.
Timing is an important element of any impact evaluation. Conducted too early, the intervention has not had time to create observable change. Conducted too late, it could reduce the usefulness of insights for informing decision-making.
Ultimately, an impact evaluation guides organizations to not only understand how impacts were generated by activities but also (based on the insights) understand what tweaks could be made to maximize effectiveness of an intervention in generating the desired outcomes.