Monitoring and Evaluation Framework
...
ICARDA implements several W1/W2, W3 and bilateral projects. Wheras every project is designed to meet particular donor requirements, such as reporting against the status of the Plan of Work and Budget (POWB) or performance indicators derived from donor results frameworks, it is difficults to make inference on the level of contribution the projects make towards the achievement of the ICARDA Strategic Plan 2017-2026. To bridge this gap, the M&E Unit initiated the development and mainstreaming of consistent and actionable indicators linked to each of the strategic research priorities and cross-cutting themes in the ICARDA Strategic Plan 2017-2026. The indicators have the unique potential of enabling aggregation of results from various projects, provided they are measured consistently across the projects.
The Indicators Reference Manual (IRM) is formulated through a consultative process. Scientists were invoved in determining which SRPs/CCTs their teams contribute to. Subsequently, the broad consultations were held with teams by their SRP/CCT groups. A presentation of the draft indicators was made to the Program Committee (PC) of the Board of Trustees in November 2019, and the PC strongly endorsed the initiative.
The indicators will be incorporated into the indicators module of the MEL Platform. Scientists will be able to adopt/retrieve institutional indicators, link customized indicators to institutional indicators, but also have the room to define idiosyncratic indicators to meet individual project requirements. The adopted/linked indicators will provide an avenue for consistent results measurement and reporting of strategic results across ICARDA projects.
...
Indicator ID | INDICATORS | SRP 1 | SRP 2 | SRP 3 | SRP 4 | SRP 5 | CCT 1 | CCT 2 | CCT 3 | CCT 4 |
---|---|---|---|---|---|---|---|---|---|---|
PR-1 | Monetary value of projects/programs/operations | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
PR-2 | Number of partnerships in which ICARDA is a party | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
PR-3 | Number of accessions in long-term storage and safety duplicated at two levels | ✔ | ||||||||
PR-4 | Number germplasm requests received | ✔ | ||||||||
PR-5 | Share of international nurseries recipients reporting performance trial results | ✔ | ||||||||
OP-1 | Number of research and development innovations | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
OP-2 | Number of ICARDA research papers published in peer reviewed journals | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
OP-3 | Number of datasets generated by ICARDA scientists | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
OP-4 | Number of people trained/ Number of people attending capacity development events | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | |
OP-5 | Number of accessions distributed | ✔ | ||||||||
OP-6 | Number of International Nurseries distributed | ✔ | ✔ | |||||||
OP-7 | Number of farm households reached with material technology packages | ✔ | ✔ | ✔ | ✔ | |||||
OP-8.1 | Quantity of seed/planting material of improved varieties accessed by farm households | ✔ | ✔ | ✔ | ✔ | |||||
OP-8.2 | Number of improved livestock breeds accessed by farm households | ✔ | ✔ | ✔ | ✔ | |||||
Outcome Indicators | ||||||||||
OC-1 | Number of research and development innovations | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
OC-2 | Alternative Metrics ('Altmetric') Score for ICARDA publications | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
OC-3 | Number of ICARDA-origin crop varieties released by national partners | ✔ | ✔ | |||||||
OC-4.1 | Number of farm households that have adopted improved crop varieities and management practices | ✔ | ✔ | ✔ | ✔ | |||||
OC-4.2 | Number of farm households that have adopted improved livestock breeds and animal husbandry practices | ✔ | ✔ | ✔ | ✔ | |||||
OC-5 | Number of policies, legal instruments or investments modified in design or implementation, informed by ICARDA research | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
OC-6 | Area under ICARDA-promoted improved crop varieties and recommended agronomic management practices | ✔ | ✔ | ✔ | ✔ | |||||
0C-7.1 | Yield of ICARDA-mandated crops | ✔ | ✔ | ✔ | ✔ | ✔ | ||||
0C-7.2 | Yield of ICARDA-mandated livestock | ✔ | ✔ | ✔ | ✔ | ✔ | ||||
OC-7.3 | Rate of yield change for ICARDA-mandated crops and livestock | ✔ | ✔ | ✔ | ✔ | ✔ | ||||
OC-8.1 | Diversity of farmers' crop production | ✔ | ✔ | ✔ | ||||||
OC-8.2 | Rangeland biodiversity | ✔ | ✔ | ✔ | ||||||
OC-8.3 | Change in rangeland ground cover | ✔ | ✔ | |||||||
OC-9.1 | Share of the improved breeds in total livestock population | ✔ | ✔ | ✔ | ||||||
OC-9.2 | Share of livestock under improved livestock managment | ✔ | ✔ | ✔ | ||||||
OC-10.1 | Consumption frequency of ICARDA-mandated crops and livestock | ✔ | ✔ | ✔ | ✔ | |||||
OC-10.2 | Quantity of ICARDA-mandated crops and livestock food products consumed | ✔ | ✔ | ✔ | ✔ | |||||
OC-10.3 | Percent of people consuming a diet of minimum diversity | ✔ | ✔ | ✔ | ✔ | |||||
OC-11.1 | Rain Use Efficiency | ✔ | ✔ | ✔ | ||||||
OC-11.2 | Biophysical water use productivity (Basic) | ✔ | ✔ | ✔ | ||||||
OC-11.3 | Biophysical water productivity (Advanced) | ✔ | ✔ | ✔ | ||||||
OC-11.4 | Absolute amount of water saved per hectare | ✔ | ✔ | ✔ | ||||||
OC-11.5 | Economic Water productivity (Basic) | ✔ | ✔ | ✔ | ||||||
OC-11.6 | Economic water productivity (Advanced) | ✔ | ✔ | ✔ | ||||||
OC-12.1 | Nutrient use efficiency (Basic) | ✔ | ✔ | |||||||
OC-12.2 | Nutrient use efficiency (Advanced) | ✔ | ✔ | |||||||
OC-13.1 | Change in soil macro-nutrient content | ✔ | ✔ | ✔ | ||||||
OC-13.2 | Change in soil micro-nutrient | ✔ | ✔ | ✔ | ||||||
OC-13.3 | Change in soil water content | ✔ | ✔ | ✔ | ||||||
OC-13.4 | Share of farmers that perceive a change in soil attributes or water use efficiency due to adoption of the improved technologies | ✔ | ✔ | ✔ | ||||||
OC-14 | Change in net income due to adoption of the improved technologies | ✔ | ✔ | ✔ | ✔ |
Impact Indicators | ||||||||||
IP-1 | Quantity of greenhouse gas emissions | ✔ | ||||||||
IP-2 | Number of households assisted to exit poverty by the interventions | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | |||
IP-3 | Number of people moved from inadequate to adequate intake of micronutrients | ✔ | ✔ |
IP-2: Number of households assisted to exit poverty by the interventions
Definition: Number of households whose per capita income has been increased to surpass the poverty line.
Unit of Measure: Count of households
Disaggregated by: Poverty line, ggeographic location (national, sub-national), gender of the household head (male, female)
Poverty lines: $1.25 per capita per day, $2 per capita per day, other nationally-defined poverty line
Method of Calculation:
At baseline:
At subsequent/endline evaluation/study:
ie-ib=αX+βI+ε
I is the intervention dummy, and the value of β is the contribution of the intervention to the increase in per capita income.
Data sources: Intervention and non-intervention households
Data collection method: Household survey with household income and expenditure module
Data collection and reporting responsibility: M&E Unit, SEP team
Data Collection and Reporting Frequency: Baseline, endline
Evidence required:
For internal evaluation or research studies: Study protocol, data collection tools, dataset, report, re-executable data management and analysis file;
For external evaluation or research studies: Request for proposals (RFP) document, inception report, final report, dataset, re-executable data management and analysis file
Rationale: Contribution towards poverty reduction is within the mission of ICARDA, and this aligns with the CGIAR Strategy and Results Framework and the United Nations' Sustainable Development Goal 1 (No Poverty). This indicator will therefore quantify ICARDA's contribution.
A major pitfall to evaluation approach is the ethics related to excluding a section of potentail beneficiaries from the intervention to allow for evaluation of the impact of the intervention on poverty. Project managers are advised to plan interventions for the control group, to be delivered after the endline/impact evaluation; and as such the evaluation ought to be carriedout within a few months to allow for intervetions to be delivered to the control group.
IP-3: Number of people moved from inadequate to adequate intake of micronutrients |
Definition: This indicator measures the number of people living in households whose suppy of micronutrients (and macronutrients) has surpassed the aggregate Recommended Dietary Allowance (RDA) and/or Estimated Average Requirement (EAR) through the consumption of ICARDA promoted/supplied crops and livestock cultivated/reared within the household or purchased from the market. |
Unit of Measure: Count of people/individuals |
Disaggregated by: Threshold (RDA, EAR), Gender, Geographic location |
Method of Calculation: |
Data sources: Intervention Households |
Data collection method: Survey with an elaborate food intake module |
Data collection and reporting responsibility: M&E Unit, SEP team |
Data Collection and Reporting Frequency: Baseline, endline'impact evaluation |
Evidence required: |
Rationale: This indicator will track the work of ICARDA is combating hunger and malnutrition, as envisaged in SLO2 and SDG 2. |
Comments and limitations: The adult equivalent approach assumes intra-household distribution of food based on metabolic energy requirements of the household members. There maybe cases where this assumption does not hold, but the cost and rigor required to conduct a study on intra-household food allocation make this a prgmatic apprach and it has been used widely in Scientific literature. In addition, the inference methods used require that the study sample is representative of (drawn randomly from) the intervention group. Careful timing of the study is also key to the validity and representativeness of the results due to temporal fluctuations in households consumption patterns. |