Monitoring and Evaluation Framework
Introduction
ICARDA implements several W1/W2, W3 and bilateral projects. Wheras every project is designed to meet particular donor requirements, such as reporting against the status of the Plan of Work and Budget (POWB) or performance indicators derived from donor results frameworks, it is difficults to make inference on the level of contribution the projects make towards the achievement of the ICARDA Strategic Plan 2017-2026. To bridge this gap, the M&E Unit initiated the development and mainstreaming of consistent and actionable indicators linked to each of the strategic research priorities and cross-cutting themes in the ICARDA Strategic Plan 2017-2026. The indicators have the unique potential of enabling aggregation of results from various projects, provided they are measured consistently across the projects.
The Indicators Reference Manual (IRM) is formulated through a consultative process. Scientists were invoved in determining which SRPs/CCTs their teams contribute to. Subsequently, the broad consultations were held with teams by their SRP/CCT groups. A presentation of the draft indicators was made to the Program Committee (PC) of the Board of Trustees in November 2019, and the PC strongly endorsed the initiative.
The indicators will be incorporated into the indicators module of the MEL Platform. Scientists will be able to adopt/retrieve institutional indicators, link customized indicators to institutional indicators, but also have the room to define idiosyncratic indicators to meet individual project requirements. The adopted/linked indicators will provide an avenue for consistent results measurement and reporting of strategic results across ICARDA projects.
List of Indicators
The indicators are categorized into 4 indicator levels (groups) corresponding to the segments of the impact pathway i.e. activity/process indicators (variables for tracking planned actions/tasks), output indicators (variables for tracking the immediate results from the activities that ICARDA and its partners carry out), outcome indicators (variables for tracking the immediate effects resulting from the use of outputs delivered by ICARDA and its partners) and impact indicators (variables for measuring or estimating the long-term results). Table 1 contains a matrix of indicators that the M&E Unit has designed for the corresponding SRPs/CCTs. The indicators include 5 process/activity indicators, 9 output-level indicators, 31 outcome-level indicators, and 3 impact-level indicators and (Figure 1).
Figure 1: Illustration of the number of indicators and the result levels they are associated with
Table 1: List of Indicators and the corresponding SRPs/CCTs
Indicator ID | INDICATORS | SRP 1 | SRP 2 | SRP 3 | SRP 4 | SRP 5 | CCT 1 | CCT 2 | CCT 3 | CCT 4 |
---|---|---|---|---|---|---|---|---|---|---|
PR-1 | Monetary value of projects/programs/operations | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
PR-2 | Number of partnerships in which ICARDA is a party | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
PR-3 | Number of accessions in long-term storage and safety duplicated at two levels | ✔ | ||||||||
PR-4 | Number germplasm requests received | ✔ | ||||||||
PR-5 | Share of international nurseries recipients reporting performance trial results | ✔ | ||||||||
OP-1 | Number of research and development innovations | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
OP-2 | Number of ICARDA research papers published in peer reviewed journals | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
OP-3 | Number of datasets generated by ICARDA scientists | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
OP-4 | Number of people trained/ Number of people attending capacity development events | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | |
OP-5 | Number of accessions distributed | ✔ | ||||||||
OP-6 | Number of International Nurseries distributed | ✔ | ✔ | |||||||
OP-7 | Number of farm households reached with material technology packages | ✔ | ✔ | ✔ | ✔ | |||||
OP-8.1 | Quantity of seed/planting material of improved varieties accessed by farm households | ✔ | ✔ | ✔ | ✔ | |||||
OP-8.2 | Number of improved livestock breeds accessed by farm households | ✔ | ✔ | ✔ | ✔ | |||||
Outcome Indicators | ||||||||||
OC-1 | Number of research and development innovations | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
OC-2 | Alternative Metrics ('Altmetric') Score for ICARDA publications | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
OC-3 | Number of ICARDA-origin crop varieties released by national partners | ✔ | ✔ | |||||||
OC-4.1 | Number of farm households that have adopted improved crop varieities and management practices | ✔ | ✔ | ✔ | ✔ | |||||
OC-4.2 | Number of farm households that have adopted improved livestock breeds and animal husbandry practices | ✔ | ✔ | ✔ | ✔ | |||||
OC-5 | Number of policies, legal instruments or investments modified in design or implementation, informed by ICARDA research | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
OC-6 | Area under ICARDA-promoted improved crop varieties and recommended agronomic management practices | ✔ | ✔ | ✔ | ✔ | |||||
0C-7.1 | Yield of ICARDA-mandated crops | ✔ | ✔ | ✔ | ✔ | ✔ | ||||
0C-7.2 | Yield of ICARDA-mandated livestock | ✔ | ✔ | ✔ | ✔ | ✔ | ||||
OC-7.3 | Rate of yield change for ICARDA-mandated crops and livestock | ✔ | ✔ | ✔ | ✔ | ✔ | ||||
OC-8.1 | Diversity of farmers' crop production | ✔ | ✔ | ✔ | ||||||
OC-8.2 | Rangeland biodiversity | ✔ | ✔ | ✔ | ||||||
OC-8.3 | Change in rangeland ground cover | ✔ | ✔ | |||||||
OC-9.1 | Share of the improved breeds in total livestock population | ✔ | ✔ | ✔ | ||||||
OC-9.2 | Share of livestock under improved livestock managment | ✔ | ✔ | ✔ | ||||||
OC-10.1 | Consumption frequency of ICARDA-mandated crops and livestock | ✔ | ✔ | ✔ | ✔ | |||||
OC-10.2 | Quantity of ICARDA-mandated crops and livestock food products consumed | ✔ | ✔ | ✔ | ✔ | |||||
OC-10.3 | Percent of people consuming a diet of minimum diversity | ✔ | ✔ | ✔ | ✔ | |||||
OC-11.1 | Rain Use Efficiency | ✔ | ✔ | ✔ | ||||||
OC-11.2 | Biophysical water use productivity (Basic) | ✔ | ✔ | ✔ | ||||||
OC-11.3 | Biophysical water productivity (Advanced) | ✔ | ✔ | ✔ | ||||||
OC-11.4 | Absolute amount of water saved per hectare | ✔ | ✔ | ✔ | ||||||
OC-11.5 | Economic Water productivity (Basic) | ✔ | ✔ | ✔ | ||||||
OC-11.6 | Economic water productivity (Advanced) | ✔ | ✔ | ✔ | ||||||
OC-12.1 | Nutrient use efficiency (Basic) | ✔ | ✔ | |||||||
OC-12.2 | Nutrient use efficiency (Advanced) | ✔ | ✔ | |||||||
OC-13.1 | Change in soil macro-nutrient content | ✔ | ✔ | ✔ | ||||||
OC-13.2 | Change in soil micro-nutrient | ✔ | ✔ | ✔ | ||||||
OC-13.3 | Change in soil water content | ✔ | ✔ | ✔ | ||||||
OC-13.4 | Share of farmers that perceive a change in soil attributes or water use efficiency due to adoption of the improved technologies | ✔ | ✔ | ✔ | ||||||
OC-14 | Change in net income due to adoption of the improved technologies | ✔ | ✔ | ✔ | ✔ |
Impact Indicators | ||||||||||
IP-1 | Quantity of greenhouse gas emissions | ✔ | ||||||||
IP-2 | Number of households assisted to exit poverty by the interventions | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | |||
IP-3 | Number of people moved from inadequate to adequate intake of micronutrients | ✔ | ✔ |
IP-2: Number of households assisted to exit poverty by the interventions |
Definition: Number of households whose per capita income has been increased to surpass the poverty line. |
Unit of Measure: Count of households |
Disaggregated by: Poverty line, ggeographic location (national, sub-national), gender of the household head (male, female) |
Method of Calculation:
|
Data sources: Intervention and non-intervention households |
Data collection method: Household survey with household income and expenditure module |
Data collection and reporting responsibility: M&E Unit, SEP team |
Data Collection and Reporting Frequency: Baseline, endline |
Evidence required: |
Rationale: Contribution towards poverty reduction is within the mission of ICARDA, and this aligns with the CGIAR Strategy and Results Framework and the United Nations' Sustainable Development Goal 1 (No Poverty). This indicator will therefore quantify ICARDA's contribution. |
Comments and limitations: Measuring data for this indicator requires early planning for the evaluation such that data from both intervention and non-intervention households is collected both at baseline and endline, at requires careful impementation of interventions to ensure that a control group of sufficient size is maintained to enable endline evaluation. In addition, the project staff need to work jointly with the M&E team to ensure that the control group is not contaminated in the course of project delivery. |
IP-3: Number of people moved from inadequate to adequate intake of micronutrients |
Definition: This indicator measures the number of people living in households whose suppy of micronutrients (and macronutrients) has surpassed the aggregate Recommended Dietary Allowance (RDA) and/or Estimated Average Requirement (EAR) through the consumption of ICARDA promoted/supplied crops and livestock cultivated/reared within the household or purchased from the market. |
Unit of Measure: Count of people/individuals |
Disaggregated by: Threshold (RDA, EAR), Gender, Geographic location |
Method of Calculation: |
Data sources: Intervention Households |
Data collection method: Survey with an elaborate food intake module |
Data collection and reporting responsibility: M&E Unit, SEP team |
Data Collection and Reporting Frequency: Baseline, endline'impact evaluation |
Evidence required: |
Rationale: This indicator will track the work of ICARDA is combating hunger and malnutrition, as envisaged in SLO2 and SDG 2. |
Comments and limitations: The adult equivalent approach assumes intra-household distribution of food based on metabolic energy requirements of the household members. There maybe cases where this assumption does not hold, but the cost and rigor required to conduct a study on intra-household food allocation make this a prgmatic apprach and it has been used widely in Scientific literature. In addition, the inference methods used require that the study sample is representative of (drawn randomly from) the intervention group. Careful timing of the study is also key to the validity and representativeness of the results due to temporal fluctuations in households consumption patterns. |