{"id":223780,"date":"2024-10-20T21:20:09","date_gmt":"2024-10-20T19:20:09","guid":{"rendered":"https:\/\/fundacionexe.org.co\/toolkit-guided-material-evaluation\/"},"modified":"2025-02-03T22:04:56","modified_gmt":"2025-02-03T21:04:56","slug":"toolkit-guided-material-evaluation","status":"publish","type":"page","link":"https:\/\/fundacionexe.org.co\/en\/toolkit-guided-material-evaluation\/","title":{"rendered":"Toolkit Guided material – Evaluation"},"content":{"rendered":"

[vc_section full_width=”stretch_row” pix_over_visibility=”” css=”.vc_custom_1728419995112{margin-top: -30px !important;background-color: #ffffff !important;}” b_custom_height=”300px”][vc_row full_width=”stretch_row” content_placement=”middle” pix_particles_check=”” css=”.vc_custom_1728420016981{margin-top: -30px !important;}”][vc_column]

\"\"<\/a>\"\"<\/a>\"\"<\/a><\/div>
<\/svg>Select<\/span><\/span><\/div><\/div><\/div>[\/vc_column][\/vc_row][\/vc_section][vc_row full_width=”stretch_row” pix_particles_check=”” css=”.vc_custom_1729546033489{margin-top: 15px !important;padding-top: 40px !important;padding-bottom: 40px !important;background-color: #14354e !important;}”][vc_column]

EVALUATION <\/span><\/span> <\/h4><\/div><\/div>[\/vc_column][\/vc_row][vc_section full_width=”stretch_row” full_height=”yes” content_placement=”middle” pix_over_visibility=”” responsive_css=”{“pix_res_sm_pt“:“30“,“pix_res_sm_pb“:“30“,“pix_res_md_pt“:“60“,“pix_res_md_pb“:“60“}” css=”.vc_custom_1728951738488{padding-top: 20px !important;padding-bottom: 20px !important;background-color: #f8f9fa !important;}”][vc_row content_placement=”middle” pix_particles_check=””][vc_column css=”.vc_custom_1592348346180{padding-top: 30px !important;padding-bottom: 30px !important;}”]

The last stage of the analysis is evaluating program effects after implementation and\/or if completion is underway. This evaluation usually includes three components: i) results evaluation, ii) impact evaluation, and iii) cost-effectiveness analysis.<\/span><\/p><\/div>

This stage intends to answer five fundamental questions about the program:<\/span><\/p><\/div>

01.+To+what+extent+have+the+objectives+been+met?<\/span><\/div>
02.+Are+the+observed+results+attributable+to+the+program?<\/span><\/div>
03.+Are+there+individual+attributes+that+moderate+program+effects?<\/span><\/div>
04.+Given+the+program's+effects,+does+the+program+have+a+reasonable+cost+to+sustain+and\/or+expand+it?<\/span><\/div>
05.+What+components+of+the+program+can+be+improved?+
<\/span><\/div><\/div><\/div>

Answering these questions not only serves as an evaluation of what has happened but also provides valuable information for the project\u2019s continuity and application in various contexts.<\/span><\/p><\/div><\/div><\/div>[\/vc_column][\/vc_row][vc_row content_placement=”middle” pix_particles_check=”” css=”.vc_custom_1729453709575{margin-top: -30px !important;}”][vc_column width=”1\/2″ css=”.vc_custom_1592348346180{padding-top: 30px !important;padding-bottom: 30px !important;}”]

<\/svg>Download the complete explanation of Stage 5 here<\/span><\/a><\/div><\/div>[\/vc_column][vc_column width=”1\/2″]
<\/svg>Download the Let\u2019s All Learn to Read Case Stage 5 here<\/span><\/a><\/div><\/div>[\/vc_column][\/vc_row][\/vc_section][vc_section full_width=”stretch_row” pix_over_visibility=”” css=”.vc_custom_1729010387596{padding-right: 50px !important;padding-left: 40px !important;background-color: #f8f9fa !important;}”][vc_row full_width=”stretch_row” pix_particles_check=”” css=”.vc_custom_1728931384425{background-color: #f8f9fa !important;}”][vc_column]
Results evaluation<\/span><\/a><\/div>
Results evaluation<\/span><\/a><\/div>
Impact evaluation<\/span><\/a><\/div>
Cost-effectiveness analysis<\/span><\/a><\/div><\/div><\/div><\/div>
Results evaluation<\/h5><\/div>
What is an evaluation?<\/span><\/span><\/span><\/div>[vc_column_text css=”.vc_custom_1729454090582{margin-top: 20px !important;}”]<\/p>\n
\n
Evaluations are periodic and objective assessments of a planned or ongoing project, policy, or program. They are also a comprehensive process of observing, collecting, measuring, systematically analyzing, and interpreting information about a program’s performance and effectiveness, and communicating information that provides inputs for policy formulation and decision-making1<\/sup><\/a>.<\/h6>\n

 <\/p>\n

There are different types of evaluations varying in program components. Some evaluate design, some outcomes, and some evaluate both. Evaluation provides an in-depth understanding of an intervention, whether public or private, which allows an evidence-based assessment of its design, implementation, results, and impact. Therefore, evaluation is essential for achieving the objective of formulating policies based on empirical evidence.<\/h6>\n<\/div>\n

[\/vc_column_text]

\"Image<\/div><\/div>
What is the evaluation for?<\/span><\/span><\/span><\/div>[vc_column_text css=”.vc_custom_1729454208190{margin-top: 20px !important;}”]<\/p>\n
\n
Evaluation guides decision-making for an efficient and effective allocation of resources and helps to identify obstacles.<\/h6>\n<\/div>\n

[\/vc_column_text][vc_row_inner][vc_column_inner]

\"In<\/div>
In terms of accountability, transparency, and negotiation, it is useful to:<\/h6>
<\/div><\/div><\/div>
\"\"<\/div>
<\/h6>
– Know how resources are being used and whether planned objectives are being achieved.<\/div><\/div><\/div>
\"\"<\/div>
<\/h6>
– Guide investment decisions.<\/div><\/div><\/div>[\/vc_column_inner][\/vc_row_inner][vc_row_inner css=”.vc_custom_1729454986337{margin-top: 20px !important;}”][vc_column_inner]
\"In<\/div>
In terms of improving the design and operation of investment programs and projects, evaluation is useful to:<\/h6>
<\/div><\/div><\/div>
\"\"<\/div>
<\/h6>
– Generate greater knowledge about the type of investments needed to obtain better results.<\/div><\/div><\/div>
\"\"<\/div>
<\/h6>
– Determine if what is being done works and the mechanisms that lead to this result.<\/div><\/div><\/div>
\"\"<\/div>
<\/h6>
– Optimizar el proceso de operaci\u00f3n de un programa o proyecto.<\/div><\/div><\/div>[vc_separator color=”custom” css=”” accent_color=”rgba(0,0,0,0.01)”][vc_separator color=”custom” css=”” accent_color=”rgba(0,0,0,0.01)”]
Factors to keep in mind<\/span><\/span><\/span><\/div>[vc_column_text css=”.vc_custom_1729455072043{margin-top: 20px !important;}”]<\/p>\n
\n
During evaluation, we must consider the four previous stages: diagnosis, design, follow-up and monitoring, and implementation. This makes it possible to verify whether the objectives have been met and the relevant answers have been addressed.<\/h6>\n<\/div>\n

[\/vc_column_text][\/vc_column_inner][\/vc_row_inner][vc_row_inner css=”.vc_custom_1729449006011{margin-top: 20px !important;}”][vc_column_inner]

<\/h5>
To identify if you have met these factors in your program design, please complete the self-evaluation survey.<\/div><\/div>
<\/svg>Self-evaluation survey.<\/span><\/a><\/div><\/div><\/div>
Types of evaluation<\/span><\/span><\/span><\/div>[vc_column_text css=”.vc_custom_1729455324169{margin-top: 20px !important;}”]<\/p>\n
\n
Two types are discussed here: results evaluation and impact evaluation.<\/h6>\n<\/div>\n

[\/vc_column_text][\/vc_column_inner][\/vc_row_inner][vc_row_inner css=”.vc_custom_1729445943063{margin-top: -10px !important;}”][vc_column_inner width=”1\/2″]

Results evaluation: Responds to descriptive questions<\/b>, about what is being executed.<\/h6>
<\/div><\/div><\/div><\/div>[\/vc_column_inner][vc_column_inner width=”1\/2″]
Impact evaluation: Responds to questions of causality<\/b>, i.e., to what extent does the program or project directly affect the desired outcome.<\/h6>
<\/div><\/div><\/div><\/div>[\/vc_column_inner][\/vc_row_inner][vc_column_text css=”.vc_custom_1729455549994{margin-top: 20px !important;}”]<\/p>\n
\n
This stage also addresses the cost-effectiveness analysis, albeit not being a type of evaluation.<\/h6>\n<\/div>\n

[\/vc_column_text]<\/div>

Results evaluation<\/h4><\/div>
What is a results evaluation?<\/span><\/span><\/span><\/div>[vc_column_text css=”.vc_custom_1729456795709{margin-top: 20px !important;}”]<\/p>\n
\n
Results evaluation focuses on the gross effects of the program, i.e., the extent to which the specific objectives of the program or project are met without discounting the influence of external factors. It allows us to measure and monitor the program results in the short and long term.<\/h6>\n

 <\/p>\n

In this type of evaluation, it is necessary to differentiate between the goods or services delivered and the results.<\/h6>\n

 <\/p>\n

The results evaluation examines:<\/h6>\n<\/div>\n

[\/vc_column_text][vc_row_inner][vc_column_inner width=”1\/2″]

The changes in the beneficiaries' conditions after a certain time of exposition to the intervention.<\/h6>
<\/div><\/div><\/div><\/div>[\/vc_column_inner][vc_column_inner width=”1\/2″]
The contributions generated by the organization for a greater result.<\/h6>
<\/div><\/div><\/div><\/div>[\/vc_column_inner][\/vc_row_inner][vc_row_inner][vc_column_inner width=”1\/2″]
The factors that affect the given outcome in the exposed beneficiaries.<\/h6>
<\/div><\/div><\/div><\/div>
How to carry out a results evaluation?<\/span><\/span><\/span><\/div>[\/vc_column_inner][vc_column_inner width=”1\/2″]
Contributions generated by alliances with partners to change the outcome.<\/h6>
<\/div><\/div><\/div><\/div>[\/vc_column_inner][\/vc_row_inner][vc_column_text css=”.vc_custom_1729457082713{margin-top: 20px !important;}”]<\/p>\n
\n
A results evaluation implies measuring the changes perceived by beneficiaries over time.<\/h6>\n

 <\/p>\n

Measuring program results is done by comparing the indicator of interest before (baseline) and after (follow-up) the intervention.<\/h6>\n

 <\/p>\n

To carry out the measurement we must:<\/h6>\n<\/div>\n

[\/vc_column_text][vc_row_inner css=”.vc_custom_1729445943063{margin-top: -10px !important;}”][vc_column_inner width=”1\/2″]

\"Establish<\/div>
Establish specific objectives<\/h6>
<\/div><\/div><\/div>
\"Stepwise<\/div>
Stepwise randomization<\/h6>
<\/div><\/div><\/div>
\"Determine<\/div>
Determine the criteria to be evaluated<\/h6>
<\/div><\/div><\/div><\/div><\/div>[\/vc_column_inner][vc_column_inner width=”1\/2″]
\"Conduct<\/div>
Conduct periodic evaluations<\/h6>
<\/div><\/div><\/div>
\"Use<\/div>
Use the results of the evaluation<\/h6>
<\/div><\/div><\/div><\/div><\/div>[\/vc_column_inner][\/vc_row_inner][vc_column_text css=”.vc_custom_1729457374114{margin-top: 20px !important;}”]<\/p>\n
\n
Remember that the factors that can influence the result are multiple, complex, interrelated, and vary constantly. The following are the steps to identify these factors3<\/sup><\/a>:<\/h6>\n<\/div>\n

[\/vc_column_text][vc_row_inner css=”.vc_custom_1729445943063{margin-top: -10px !important;}”][vc_column_inner width=”1\/2″]

01.<\/span><\/div>
Collect and analyze information<\/h6>
<\/div><\/div><\/div><\/div><\/div>[\/vc_column_inner][vc_column_inner width=”1\/2″]
02.<\/span><\/div>
Identify the most important contributing factors that drive change<\/h6>
<\/div><\/div><\/div><\/div><\/div>[\/vc_column_inner][\/vc_row_inner][vc_row_inner css=”.vc_custom_1729445943063{margin-top: -10px !important;}”][vc_column_inner width=”1\/2″]
03.<\/span><\/div>
Review local sources of knowledge about the factors influencing the result<\/h6>
<\/div><\/div><\/div><\/div><\/div>[\/vc_column_inner][vc_column_inner width=”1\/2″]
04.<\/span><\/div>
Solve the problem of the results\u2019 unintended second-order effects<\/h6>
<\/div><\/div><\/div><\/div><\/div>[\/vc_column_inner][\/vc_row_inner][vc_column_text css=”.vc_custom_1729457751960{margin-top: 20px !important;}”]<\/p>\n
\n
When assessing results, it is crucial to consider the organization\u2019s contributions and those arising from partnerships to modify the outcome.<\/h6>\n

 <\/p>\n

<\/h6>\n
Organizational contributions<\/strong> span the full range of activities and initiatives, within and outside projects4<\/sup><\/a>. These contributions can be established by:<\/h6>\n<\/div>\n

[\/vc_column_text][vc_row_inner][vc_column_inner]

A.<\/span><\/div>
Analyzing the consistency between the organization\u2019s strategy and the overall operational management about changes in results.<\/h6>
<\/div><\/div><\/div>
B.<\/span><\/div>
Verifying if the organization's planning and intervention management are aligned to leverage synergies contributing to results.<\/h6>
<\/div><\/div><\/div>
C.<\/span><\/div>
Determining whether or not individual outputs are effective in contributing to results.<\/h6>
<\/div><\/div><\/div>[vc_column_text css=”.vc_custom_1729458442076{margin-top: 20px !important;}”]<\/p>\n
\n
Results are influenced by a complex range of factors. Change invariably requires the joint action of several institutions and stakeholders5<\/sup><\/a>. The contributions generated by partnerships<\/strong> can be established by:<\/h6>\n<\/div>\n

[\/vc_column_text][\/vc_column_inner][\/vc_row_inner][vc_row_inner css=”.vc_custom_1729445943063{margin-top: -10px !important;}”][vc_column_inner width=”1\/3″]

Examining the degree of mutual support between partners<\/h6>
<\/div><\/div><\/div><\/div>[\/vc_column_inner][vc_column_inner width=”1\/3″]
Examining how partnerships were formed and their performance.<\/h6>
<\/div><\/div><\/div><\/div>[\/vc_column_inner][vc_column_inner width=”1\/3″]
Conducting joint evaluations<\/h6>
<\/div><\/div><\/div><\/div>[\/vc_column_inner][\/vc_row_inner]<\/div>

Impact evaluation<\/h4><\/div>
What is an impact evaluation?<\/span><\/span><\/span><\/div>[vc_column_text css=”.vc_custom_1729458977339{margin-top: 20px !important;}”]<\/p>\n
\n
Impact evaluation is a type of evaluation focused on answering questions related to the causal relationship between a result and the specific program or project. It aims to identify observable changes in the result that can be directly and uniquely attributed to the evaluated program or project.<\/h6>\n<\/div>\n

[\/vc_column_text]

Questions that an impact evaluation seeks to answer<\/span><\/span><\/span><\/div>[vc_column_text css=”.vc_custom_1729459093160{margin-top: 20px !important;}”]<\/p>\n
\n
All impact evaluations answer a cause-effect question.<\/h6>\n
.<\/h6>\n
These questions can be applied to any setting, as long as they assess the impact of a program modality or design innovation, not just the program itself.<\/h6>\n

 <\/p>\n

Here are some examples of questions that an impact evaluation might address:<\/h6>\n<\/div>\n

[\/vc_column_text][vc_row_inner css=”.vc_custom_1729459667905{margin-top: -20px !important;}”][vc_column_inner width=”1\/2″]

What is the causal effect (impact) of providing scholarships to students for school attendance and academic performance?<\/h6>
<\/div><\/div><\/div><\/div>[\/vc_column_inner][vc_column_inner width=”1\/2″]
Does improving roads and access roads increase access to the labor market and boost household incomes?<\/h6>
<\/div><\/div><\/div><\/div>[\/vc_column_inner][\/vc_row_inner][vc_row_inner][vc_column_inner width=”1\/2″]
How does replacing dirt floors with concrete floors impact children's health?<\/h6>
<\/div><\/div><\/div><\/div>[\/vc_column_inner][vc_column_inner width=”1\/2″]
Does class size affect student learning?<\/h6>
<\/div><\/div><\/div><\/div>[\/vc_column_inner][\/vc_row_inner][vc_column_text css=”.vc_custom_1729459970575{margin-top: 20px !important;}”]<\/p>\n
\n
To answer cause-effect<\/strong> questions, it is necessary to understand the concept of potential or counterfactual results.<\/h6>\n

 <\/p>\n

The counterfactual is the hypothetical scenario<\/strong> that would have occurred in the absence of the program. Particularly, to see the treatment effect, the counterfactual is the result observed in the scenario in which the program participants would not have participated in the program.<\/h6>\n

 <\/p>\n

While it is not possible to observe a counterfactual for each individual, it is possible to construct counterfactuals if our objective is to analyze averages. Moreover, this is why when we talk about the effect of a program or intervention, we usually talk about the average treatment effect.<\/h6>\n

 <\/p>\n

Usually, to achieve the correspondence between the two groups, we use statistical models that depend on the program design and the targeting method used in the program. Thus, depending on targeting method, impact evaluation designs are classified as experimental or non-experimental6<\/sup><\/a>.<\/h6>\n

 <\/p>\n

In experimental designs<\/strong>, individuals are randomly assigned to the treatment or control group. Assigning individuals to program beneficiaries in this way ensures that they are comparable in all observable and unobservable characteristics.<\/h6>\n

 <\/p>\n

Experimental designs can assign treatment in different ways depending on the situation of the project to be evaluated, always maintaining the criterion of randomization. Thus, randomization can be:<\/h6>\n<\/div>\n

[\/vc_column_text][vc_row_inner css=”.vc_custom_1729445943063{margin-top: -10px !important;}”][vc_column_inner width=”1\/2″]

\"Simple<\/div>
Simple randomization<\/h6>
<\/div><\/div><\/div>
\"Stepwise<\/div>
Stepwise randomization<\/h6>
<\/div><\/div><\/div><\/div><\/div>[\/vc_column_inner][vc_column_inner width=”1\/2″]
\"Randomization<\/div>
Randomization by strata<\/h6>
<\/div><\/div><\/div>
\"Block<\/div>
Block randomization<\/h6>
<\/div><\/div><\/div><\/div><\/div>[\/vc_column_inner][\/vc_row_inner][vc_column_text css=”.vc_custom_1729460263747{margin-top: 20px !important;}”]<\/p>\n
\n
The effects of a program with an experimental design can be obtained through the difference in the averages of the results of the treatment and control groups, once a considerable time has elapsed in the implementation of the program.<\/h6>\n

 <\/p>\n

Non-experimental designs<\/strong> are those in which treatment assignment is not randomized and, therefore, multiple observable and unobservable characteristics are related to the inclusion of individuals in the program. Given these dissimilarities, the difference in the average results of the two groups does not provide sufficient information on the impact of the intervention. It is therefore necessary to employ more robust statistical methods such as:<\/h6>\n<\/div>\n

[\/vc_column_text][vc_row_inner css=”.vc_custom_1729445943063{margin-top: -10px !important;}”][vc_column_inner width=”1\/2″]

\"Differences-in-Differences<\/div>
Differences-in-Differences (DiD)<\/h6>
<\/div><\/div><\/div>
\"Matching\"<\/div>
Matching<\/h6>
<\/div><\/div><\/div><\/div><\/div>[\/vc_column_inner][vc_column_inner width=”1\/2″]
\"Instrumental<\/div>
Instrumental variables<\/h6>
<\/div><\/div><\/div>
\"Discontinuous<\/div>
Discontinuous regression<\/h6>
<\/div><\/div><\/div><\/div><\/div>[\/vc_column_inner][\/vc_row_inner][vc_column_text css=”.vc_custom_1729460894112{margin-top: 20px !important;}”]<\/p>\n
\n
It is worth noting that the two main factors for choosing the non-experimental method are the program allocation method and information availability. Therefore, it is not feasible to apply all methods in every situation.<\/h6>\n<\/div>\n

[\/vc_column_text]

Types of impact evaluations according to time<\/span><\/span><\/span><\/div>[vc_column_text css=”.vc_custom_1729461124320{margin-top: 20px !important;}”]<\/p>\n
\n
Impact evaluations can be classified as prospective or retrospective depending on how the result variables are measured. Prospective evaluations are usually more robust in that they measure program results from the beginning of the program. Retrospective evaluations are usually conducted when there is very little information on program implementation and design.<\/h6>\n<\/div>\n

[\/vc_column_text][vc_row_inner][vc_column_inner]

An impact evaluation is much more robust if designed jointly with the program.<\/h6>
<\/div><\/div>
Design stage<\/span><\/a><\/div><\/div><\/div>[\/vc_column_inner][\/vc_row_inner]
<\/svg>Download the template for the products required for evaluation here.<\/span><\/a><\/div>
How do you decide whether or not to do an impact evaluation?<\/span><\/span><\/span><\/div>[vc_column_text css=”.vc_custom_1729461914611{margin-top: 20px !important;}”]<\/p>\n
\n
Not all programs or projects require an impact evaluation; one is carried out because you want to answer a cause-effect question that provides specific information about the program or project.<\/h6>\n

 <\/p>\n

Keep in mind that, if information is unavailable, you must collect data, and this can make the evaluation very costly.<\/h6>\n

 <\/p>\n

If you are starting a new program, or are thinking of expanding or innovating an existing one, answer a series of questions to help you decide whether to conduct an impact evaluation. The following figure shows the questions that will help determine whether or not to conduct an impact evaluation.<\/h6>\n<\/div>\n

[\/vc_column_text]

<\/svg>Download the Decision Format Template here<\/span><\/a><\/div><\/div>

Cost-effectiveness analysis<\/h4><\/div>[vc_column_text css=”.vc_custom_1729462846361{margin-top: 20px !important;}”]<\/p>\n
\n
Cost-effectiveness analysis is a method used to measure the ratio between a program or intervention\u2019s monetary cost and the results obtained during the impact evaluation stage. Accordingly, a cost-effectiveness analysis makes it possible to quantify the effectiveness obtained for each unit of additional cost by aggregating total costs and benefits.<\/h6>\n

 <\/p>\n

In addition, it is possible to compare different interventions through their cost-effectiveness analyses and, based on these, determine which program is the best alternative in financial terms.<\/h6>\n<\/div>\n

[\/vc_column_text]

Requirements for a cost-effectiveness analysis<\/span><\/span><\/span><\/div>[vc_column_text css=”.vc_custom_1729462939118{margin-top: 20px !important;}”]<\/p>\n
\n
\n
\n
\n
\n
A cost-effectiveness analysis requires detailed considerations of the impacts and costs of the relevant program.<\/h6>\n<\/div>\n<\/div>\n
\n
\n
<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n
<\/div>\n<\/div>\n<\/div>\n<\/div>\n

[\/vc_column_text][vc_row_inner css=”.vc_custom_1729459667905{margin-top: -20px !important;}”][vc_column_inner]

\"A<\/div>
A cost-effectiveness analysis requires detailed considerations of the impacts and costs of the relevant program.<\/h6>
<\/p>\n
  • Accurate estimates of the impact on all relevant results are required along with the periodicity of those effects.\n
  • A systematic review of the relevant literature should be conducted.<\/div><\/div><\/div><\/div><\/div>[\/vc_column_inner][\/vc_row_inner][vc_row_inner css=”.vc_custom_1729459667905{margin-top: -20px !important;}”][vc_column_inner]
    \"Regarding<\/div>
    Regarding program costs:<\/h6>
    \n
  • Since having retrospective information implies higher costs and margins of error, it is recommended to track program implementation costs constantly during development.\n
  • It is essential to collect and account for disaggregated costs to assess purchases, contracting, and associated costs accurately.\n
  • It is important to consider all factors that regularly affect monetary measures.<\/div><\/div><\/div><\/div><\/div>[\/vc_column_inner][\/vc_row_inner][vc_column_text css=”.vc_custom_1729463351054{margin-top: 20px !important;}”]
    \n
    \n
    A program cost detailed analysis should identify:<\/h6>\n<\/div>\n

    [\/vc_column_text][vc_row_inner][vc_column_inner width=”1\/2″]

    01.<\/span><\/div>
    List of the components and materials required for program implementation<\/h6>
    <\/div><\/div><\/div><\/div>[\/vc_column_inner][vc_column_inner width=”1\/2″]
    02.<\/span><\/div>
    Quantities and unit costs of these materials<\/h6>
    <\/div><\/div><\/div><\/div>[\/vc_column_inner][\/vc_row_inner][vc_column_text css=”.vc_custom_1729463351054{margin-top: 20px !important;}”]<\/p>\n
    \n
    A program cost detailed analysis should identify:<\/h6>\n<\/div>\n

    [\/vc_column_text][vc_row_inner css=”.vc_custom_1729445943063{margin-top: -10px !important;}”][vc_column_inner width=”1\/2″]

    \"Program<\/div>
    Program administration<\/h6>
    <\/div><\/div><\/div>
    \"Targeting<\/div>
    Targeting costs<\/h6>
    <\/div><\/div><\/div>
    \"Staff<\/div>
    Staff training<\/h6>
    <\/div><\/div><\/div>
    \"Participant<\/div>
    Participant training<\/h6>
    <\/div><\/div><\/div><\/div><\/div>[\/vc_column_inner][vc_column_inner width=”1\/2″]
    \"Implementation<\/div>
    Implementation costs<\/h6>
    <\/div><\/div><\/div>
    \"Beneficiary<\/div>
    Beneficiary costs<\/h6>
    <\/div><\/div><\/div>
    \"Averted<\/div>
    Averted costs<\/h6>
    <\/div><\/div><\/div>
    \"Monitoring<\/div>
    Monitoring costs<\/h6>
    <\/div><\/div><\/div><\/div><\/div>[\/vc_column_inner][\/vc_row_inner]
    Ingredient method for cost-effectiveness analysis<\/span><\/span><\/span><\/div>[vc_column_text css=”.vc_custom_1729463577817{margin-top: 20px !important;}”]<\/p>\n
    \n
    In line with J-PAL, we believe it is advisable to follow the guide proposed by Dhaliwal et al. (2012) to perform cost-effectiveness analysis. This guide facilitates the collection of disaggregated program costs using an ingredient method, which consists of listing the program ingredients, estimating their unit costs, and determining their utilization through the implementation of the intervention. According to Dhaliwal et al. (2012), the costs associated with program design, evaluation, and research should not be considered in such an analysis7.<\/sup><\/a><\/h6>\n<\/div>\n

    [\/vc_column_text]

    <\/svg>Download the J-PAL costing template here<\/span><\/a><\/div>[vc_column_text css=”.vc_custom_1729463644677{margin-top: 20px !important;}”]<\/p>\n
    \n
    Once all the program costs and effects are obtained, they can be aggregated to make the program cost-effectiveness ratio as follows:<\/h6>\n<\/div>\n

    [\/vc_column_text][vc_row_inner css=”.vc_custom_1729445943063{margin-top: -10px !important;}”][vc_column_inner]

    \"Image<\/div><\/div><\/div><\/div>[\/vc_column_inner][\/vc_row_inner][vc_text_separator title=”” color=”custom” border_width=”2″ css=”” accent_color=”#13334c”][vc_column_text css=”.vc_custom_1729609170037{margin-top: 20px !important;}”]<\/p>\n
    \n
      \n
    1. Gertler, P. J., Mart\u00ednez, S., Premand, P., Rawlings, L. B. and Vermeersch, C. (2011). La evaluaci\u00f3n de impacto en la pr\u00e1ctica<\/em>. Washington D.C.: World Bank. Extracted from: https:\/\/openknowledge.worldbank.org\/entities\/publication\/f090e5a0-16f6-5795-a3b2-7f711c8b7eb7 <\/a><\/li>\n
    2. PNUD.(2002). Informe sobre desarrollo humano 2002: Profundizar la democracia en un mundo fragmentado.<\/em> Madrid\/Barcelona: Mundi-Prensa Libros. Extracted from: https:\/\/hdr.undp.org\/system\/files\/documents\/hdr2002espdf.pdf <\/a><\/li>\n
    3. Ibid<\/li>\n
    4. Ibid<\/li>\n
    5. Ibid<\/li>\n
    6. Morra-Imas, L.G. y Rist, R.C.(2009). El camino hacia los resultados: dise\u00f1o y realizaci\u00f3n de evaluaciones eficaces para el desarrollo. <\/em>Washington D.C.: World Bank.<\/li>\n
    7. Dhaliwal, I., Duflo, E., Glennerster, R. and Tulloch, C. (2012). Comparative Cost-Effectiveness Analysis to Inform Policy in Developing Countries: A General Framework with Applications for Education. <\/em>Abdul Latif Jameel Poverty Action Lab (J-PAL), MIT. Extracted from: https:\/\/www.povertyactionlab.org\/sites\/default\/files\/research-resources\/CEA%20in%20Education%202013.01.29_0.pdf <\/a><\/li>\n<\/ol>\n<\/div>\n

      [\/vc_column_text]<\/div><\/div><\/div><\/div><\/div><\/div>[\/vc_column][\/vc_row][\/vc_section][vc_section pix_over_visibility=””][vc_row pix_particles_check=””][vc_column][\/vc_column][\/vc_row][vc_row pix_particles_check=”” css=”.vc_custom_1730143170366{margin-top: 20px !important;}”][vc_column width=”1\/2″]

      \"Image<\/a><\/div>[\/vc_column][vc_column width=”1\/2″]
      \"Image<\/a><\/div>[\/vc_column][\/vc_row][\/vc_section]<\/p>\n<\/div>","protected":false},"excerpt":{"rendered":"[vc_section full_width=”stretch_row” pix_over_visibility=”” css=”.vc_custom_1728419995112{margin-top: -30px !important;background-color: #ffffff !important;}” b_custom_height=”300px”][vc_row full_width=”stretch_row” content_placement=”middle” pix_particles_check=”” css=”.vc_custom_1728420016981{margin-top: -30px !important;}”][vc_column][\/vc_column][\/vc_row][\/vc_section][vc_row full_width=”stretch_row” pix_particles_check=”” css=”.vc_custom_1729546033489{margin-top: 15px !important;padding-top: 40px !important;padding-bottom: 40px !important;background-color: #14354e !important;}”][vc_column][\/vc_column][\/vc_row][vc_section full_width=”stretch_row” full_height=”yes” content_placement=”middle” pix_over_visibility=”” responsive_css=”{“pix_res_sm_pt“:“30“,“pix_res_sm_pb“:“30“,“pix_res_md_pt“:“60“,“pix_res_md_pb“:“60“}” css=”.vc_custom_1728951738488{padding-top: 20px !important;padding-bottom: 20px !important;background-color: #f8f9fa !important;}”][vc_row content_placement=”middle” pix_particles_check=””][vc_column…","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-223780","page","type-page","status-publish","hentry"],"yoast_head":"\nToolkit Guided material - Evaluation - Fundaci\u00f3n Empresarios por la Educaci\u00f3n FExE<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/fundacionexe.org.co\/en\/toolkit-guided-material-evaluation\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Toolkit Guided material - Evaluation - Fundaci\u00f3n Empresarios por la Educaci\u00f3n FExE\" \/>\n<meta property=\"og:description\" content=\"[vc_section full_width=”stretch_row” pix_over_visibility=”” css=”.vc_custom_1728419995112{margin-top: -30px !important;background-color: #ffffff !important;}” b_custom_height=”300px”][vc_row full_width=”stretch_row” content_placement=”middle” pix_particles_check=”” css=”.vc_custom_1728420016981{margin-top: -30px !important;}”][vc_column][\/vc_column][\/vc_row][\/vc_section][vc_row full_width=”stretch_row” pix_particles_check=”” css=”.vc_custom_1729546033489{margin-top: 15px !important;padding-top: 40px !important;padding-bottom: 40px !important;background-color: #14354e !important;}”][vc_column][\/vc_column][\/vc_row][vc_section full_width=”stretch_row” full_height=”yes” content_placement=”middle” pix_over_visibility=”” responsive_css=”{“pix_res_sm_pt“:“30“,“pix_res_sm_pb“:“30“,“pix_res_md_pt“:“60“,“pix_res_md_pb“:“60“}” css=”.vc_custom_1728951738488{padding-top: 20px !important;padding-bottom: 20px !important;background-color: #f8f9fa !important;}”][vc_row content_placement=”middle” pix_particles_check=””][vc_column...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/fundacionexe.org.co\/en\/toolkit-guided-material-evaluation\/\" \/>\n<meta property=\"og:site_name\" content=\"Fundaci\u00f3n Empresarios por la Educaci\u00f3n FExE\" \/>\n<meta property=\"article:modified_time\" content=\"2025-02-03T21:04:56+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"41 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/fundacionexe.org.co\\\/en\\\/toolkit-guided-material-evaluation\\\/\",\"url\":\"https:\\\/\\\/fundacionexe.org.co\\\/en\\\/toolkit-guided-material-evaluation\\\/\",\"name\":\"Toolkit Guided material - Evaluation - Fundaci\u00f3n Empresarios por la Educaci\u00f3n FExE\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/fundacionexe.org.co\\\/en\\\/#website\"},\"datePublished\":\"2024-10-20T19:20:09+00:00\",\"dateModified\":\"2025-02-03T21:04:56+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/fundacionexe.org.co\\\/en\\\/toolkit-guided-material-evaluation\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/fundacionexe.org.co\\\/en\\\/toolkit-guided-material-evaluation\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/fundacionexe.org.co\\\/en\\\/toolkit-guided-material-evaluation\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Portada\",\"item\":\"https:\\\/\\\/www.fundacionexe.org.co\\\/en\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Toolkit Guided material – Evaluation\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/fundacionexe.org.co\\\/en\\\/#website\",\"url\":\"https:\\\/\\\/fundacionexe.org.co\\\/en\\\/\",\"name\":\"Fundaci\u00f3n Empresarios por la Educaci\u00f3n FExE\",\"description\":\"Red de conocimiento independiente. Sumamos con informaci\u00f3n y evidencia para que Colombia d\u00e9 un salto en educaci\u00f3n de calidad.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/fundacionexe.org.co\\\/en\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Toolkit Guided material - Evaluation - Fundaci\u00f3n Empresarios por la Educaci\u00f3n FExE","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/fundacionexe.org.co\/en\/toolkit-guided-material-evaluation\/","og_locale":"en_US","og_type":"article","og_title":"Toolkit Guided material - Evaluation - Fundaci\u00f3n Empresarios por la Educaci\u00f3n FExE","og_description":"[vc_section full_width=”stretch_row” pix_over_visibility=”” css=”.vc_custom_1728419995112{margin-top: -30px !important;background-color: #ffffff !important;}” b_custom_height=”300px”][vc_row full_width=”stretch_row” content_placement=”middle” pix_particles_check=”” css=”.vc_custom_1728420016981{margin-top: -30px !important;}”][vc_column][\/vc_column][\/vc_row][\/vc_section][vc_row full_width=”stretch_row” pix_particles_check=”” css=”.vc_custom_1729546033489{margin-top: 15px !important;padding-top: 40px !important;padding-bottom: 40px !important;background-color: #14354e !important;}”][vc_column][\/vc_column][\/vc_row][vc_section full_width=”stretch_row” full_height=”yes” content_placement=”middle” pix_over_visibility=”” responsive_css=”{“pix_res_sm_pt“:“30“,“pix_res_sm_pb“:“30“,“pix_res_md_pt“:“60“,“pix_res_md_pb“:“60“}” css=”.vc_custom_1728951738488{padding-top: 20px !important;padding-bottom: 20px !important;background-color: #f8f9fa !important;}”][vc_row content_placement=”middle” pix_particles_check=””][vc_column...","og_url":"https:\/\/fundacionexe.org.co\/en\/toolkit-guided-material-evaluation\/","og_site_name":"Fundaci\u00f3n Empresarios por la Educaci\u00f3n FExE","article_modified_time":"2025-02-03T21:04:56+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"41 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/fundacionexe.org.co\/en\/toolkit-guided-material-evaluation\/","url":"https:\/\/fundacionexe.org.co\/en\/toolkit-guided-material-evaluation\/","name":"Toolkit Guided material - Evaluation - Fundaci\u00f3n Empresarios por la Educaci\u00f3n FExE","isPartOf":{"@id":"https:\/\/fundacionexe.org.co\/en\/#website"},"datePublished":"2024-10-20T19:20:09+00:00","dateModified":"2025-02-03T21:04:56+00:00","breadcrumb":{"@id":"https:\/\/fundacionexe.org.co\/en\/toolkit-guided-material-evaluation\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/fundacionexe.org.co\/en\/toolkit-guided-material-evaluation\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/fundacionexe.org.co\/en\/toolkit-guided-material-evaluation\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Portada","item":"https:\/\/www.fundacionexe.org.co\/en\/"},{"@type":"ListItem","position":2,"name":"Toolkit Guided material – Evaluation"}]},{"@type":"WebSite","@id":"https:\/\/fundacionexe.org.co\/en\/#website","url":"https:\/\/fundacionexe.org.co\/en\/","name":"Fundaci\u00f3n Empresarios por la Educaci\u00f3n FExE","description":"Red de conocimiento independiente. Sumamos con informaci\u00f3n y evidencia para que Colombia d\u00e9 un salto en educaci\u00f3n de calidad.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/fundacionexe.org.co\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/fundacionexe.org.co\/en\/wp-json\/wp\/v2\/pages\/223780","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fundacionexe.org.co\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/fundacionexe.org.co\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/fundacionexe.org.co\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/fundacionexe.org.co\/en\/wp-json\/wp\/v2\/comments?post=223780"}],"version-history":[{"count":11,"href":"https:\/\/fundacionexe.org.co\/en\/wp-json\/wp\/v2\/pages\/223780\/revisions"}],"predecessor-version":[{"id":224018,"href":"https:\/\/fundacionexe.org.co\/en\/wp-json\/wp\/v2\/pages\/223780\/revisions\/224018"}],"wp:attachment":[{"href":"https:\/\/fundacionexe.org.co\/en\/wp-json\/wp\/v2\/media?parent=223780"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}