Select
Implementation process is the fourth stage of the process and aims to analyze the program management and the achievements during its execution.
To this end, five steps explain the process evaluation, including the creation of the “General Process Model” or “Process Flow Chart” tool and the “SWOT Analysis”.
Process evaluation
In this section, we will explore the concept of process evaluation and its objectives.
Definition and purpose
Process evaluation analyzes a program’s operational management to assess how and to what extent the goals and objectives are achieved. It is an exercise executed during implementation that leverages information from the monitoring and other sources to evaluate the operation comprehensively. It answers normative questions by comparing what is happening with what should ideally be happening, and examines activities and whether planned objectives are met.
Process evaluation examines the Theory of Change implementation, which, as noted in the second stage, the design stage, includes inputs, activities, outputs, and results as variables.
In the gap between the plan and the reality are the problems observed in the variables of the Theory of Change (to which the context is added), which serve to identify strengths, weaknesses, opportunities, and threats, and make recommendations for improvement, as appropriate.
Objectives of process evaluation
The key objectives of the process evaluation are:
To describe the operational management at the different levels of program implementation, analyzing whether the management and articulation of processes contribute to achieving its objectives.
To identify good practices, strengths, opportunities for improvement, weaknesses, and threats of the program.
To design general and specific recommendations that the program can implement at the normative and operational levels.
Steps to carry out a process evaluation
Follow these steps for an adequate process evaluation:
01. Desk analysis and fieldwork
This first step includes a desk analysis and fieldwork planning and execution:
Desk analysis considers the collection, organization, systematization, and evaluation of information contained in administrative records, external evaluations, review of official and regulatory documents, information systems, and program-related documents.
Fieldwork involves quantitative and qualitative information:
Fieldwork organization
This work requires additional organizational work to determine: i) the survey sample to collect information; ii) the data collection instruments to be applied; iii) the schedule of activities; iv) the fieldwork agenda, and v) the consent forms
02. Program description and elaboration of the process general model
The second step in a process evaluation consists of describing the program from its main activities based on its objectives, the problem it addresses, and the relevant context.
Determine the order of program processes
Describe in detail the development of each process of the program
Map processes
Each program element is characterized according to the designed Theory of Change (Design stage) and assigned a responsible person.
In relation to the processes, it is key to know if:
There are documents that regulate the processes.
All operators know the documented processes.
The processes are standardized, i.e., if they are used by all executing agencies.
There is a monitoring system and management indicators that provide feedback on the operational processes developed by the operators.
There are mechanisms for the systematic implementation of improvements.
The analysis aims to optimize the program’s processes. Some key aspects to consider include:
Simplify processes
Evaluate process steps and their activities according to three categories:
03. Input and process analysis
The following topics should be considered in the description and analysis of the process general model:
A.
A detailed description of the activities, components, and stakeholders involved in the development of the process
B.
The determination of the limits of the process and its relationship with other processes
C.
The available inputs and resources, considering:
- Time
- Personnel
- Financial resources
- infrastructure
D.
The outputs
E.
Information systems
F.
Coordination
G.
Evaluation of the process relevance
H.
Identification of the characteristics related to the process’ strategic importance.
I.
The existence of mechanisms to know the beneficiaries' satisfaction
04. SWOT analysis
To evaluate a program and its results, it is key to identify how to adjust the intervention in the future to strengthen its effects. The SWOT analysis (strengths, weaknesses, opportunities, and threats) is a useful tool to recognize the aspects that work adequately, the spaces and opportunities that can be explored, the elements that require improvement, and the challenges that could arise when continuing with the program. The good practices identified and the bottlenecks detected can also be included in this analysis.
When conducting the SWOT analysis, it is essential not to lose sight of some of the main reasons for program failure that were discussed in the follow-up and monitoring stage (design errors, implementation failures, and external factors). If these reasons are noted, it is important to work on them to ensure program success in the future.
05. Formulation of recommendations and conclusions
Based on the findings and results of the comprehensive and specific analysis of the program’s operational management, the process general model and/or the SWOT analysis help to make recommendations at the process and general levels.
The main purpose of the recommendations is to provide articulated and coherent lines and strategies of action for the improvement of the program’s operation. These recommendations should consider the following elements:
General description of the recommendation
Comparison between the current situation and the expected result of implementing the recommendation
Identification of the main person(s) responsible for implementation
Analysis of the feasibility of implementation
Analysis of the potential effects that implementation may have on the operation of the program and the achievement of objectives
Level of prioritization of the recommendation
