Evaluation Summary What Can We Learn from Nutrition Impact Evaluations? High levels of child malnutrition in developing countries contribute to mortality and have long-term consequences for children s cognitive development and earnings as adults. Recent impact evaluations show that many different interventions have had an impact on children s anthropometric outcomes (height, weight, and birth weight), but there is no simple answer to the question What works? to address the problem. Similar interventions have widely different results in different settings, owing to differences in local context, the causes and severity of malnutrition, and the capacity for program implementation. Impact evaluations of programs supported by the Bank, which are generally large-scale, complex inter-ventions in low-capacity settings, show equally variable results. The findings confirm that it should not be assumed that an intervention found effective in a randomized medical setting will have the same effects when implemented under field conditions. There are many robust experimental and quasi-experimental methods for assessing impact under difficult circumstances often found in field settings. The relevance and impact of nutrition impact evaluations could be enhanced by collecting data on service delivery, demand-side behavioral outcomes, and implementation processes to better understand the causal chain and what part of the chain is weak, in parallel with impact evaluations. It is also important to understand better the distribution of impacts, particularly among the poor, and to document better the costs and effectiveness of interventions. High levels of child malnutrition in developing countries are contributing to mortality and present long-term consequences for the survivors. An estimated 178 million children under age 5 in developing countries are stunted (low height for age) and 55 million are wasted (low weight for height). Malnutrition makes children more susceptible to illness and strongly affects child mortality. Beyond the mortality risk in the short run, the developmental delays caused by undernutrition affect children s cognitive outcomes and productive potential as adults. Micronutrient deficiencies vitamin A, iron, zinc, iodine, for example are also common and have significant consequences. Progress in reducing malnutrition has been slow: More than half of countries are not on track to achieve the Millennium Development Goal of halving the share of children who are malnou-rished (low weight for age) by 2015. The food price and financial crises are making achievement of this goal even more elusive. The World Bank has recently taken steps to ex-pand its support for nutrition in response to the underlying need and the increased urgency due to the crises. WHAT DO WE KNOW ABOUT REDUCING MALNUTRITION? The increased interest and resources focused on the problem of high and potentially increasing rates of undernutrition raises the question, what do we know about the causes of malnutrition and the in-terventions most likely to reduce it? The medical literature points to the need to inter-vene during gestation and the first two years of life to prevent child malnutrition and its consequences. It suggests that investments in interventions during this window of opportunity among children under 2 are likely to have the greatest benefits. Recently published meta-analyses of the impact evaluation literature point to several interventions found effective for reducing undernutrition in spe-cific settings. However, there are limitations to the generalizability of those reviews findings, particularly in the context of large-scale government programs most likely to be supported by the World Bank. The reviews tend to disproportionately draw on the findings of smaller, controlled experiments; there are few examples of evaluations of large-scale programs, over which there is less control in implementation. In reviewing a large number of studies, interventions, and outcomes, they tend to focus on average impacts. They generally do not explain the magnitude or variability of impacts across or within studies. Very few address the programmatic reasons why some interventions work or don t work, nor do they assess the cost-effectiveness of interventions. Objectives of the Review This paper reviews recent impact evaluations of interventions and programs to improve child anth-ropometric outcomes height, weight, and birth weight with an emphasis on both the findings and limitations of the literature and on understanding what might happen in a non-research setting. It further reviews in greater detail the experience and lessons from evaluations of the impact of World Bank-supported programs on nutrition outcomes. Specifically, the review addresses four questions. First, what can be said about the impact of different interventions on children s anthropometric outcomes? Second, how do these findings vary across settings and within target groups, and what accounts for this variability? Third, what is the evidence of the cost-effectiveness of these interventions? Finally, what have been the lessons from implementing impact evaluations of Bank-supported programs with anthropometric impacts? While there are different dimensions of child nutri-tion that could be explored, the report focuses on child anthropometric outcomes -- weight, height, and birth weight. These are the most common nutrition outcome indicators in the literature and the most frequently monitored by national nutrition programs supported by the World Bank. Low weight for age (underweight) is also the indicator for one of the MDGs. Methodology and Scope Forty-six nutrition impact evaluations published since 2000 were systematically reviewed. These evaluations assessed the impact of diverse interven-tions community nutrition programs, conditional and unconditional cash transfers, early child devel-opment programs, food aid, integrated health and nutrition services, and de-worming. All of the evaluations used research designs that compared the outcomes among those affected by the project to the counterfactual that is, what would have happened to a similar group of people in the absence of the intervention. About half used randomized assignment to create treatment and control groups, while the remainder used matching and various econometric techniques to construct a counterfactual. Among the 46 evaluations, twelve assessed the im-pact of World Bank-supported programs on nutri-tion outcomes in eight countries. While the broader review relies on the analysis of the published impact evaluations as the main source of data, for these twelve evaluations project documents and research outputs were reviewed and World Bank staff, country officials and the evaluators and re-searchers who conducted the studies were interviewed. Findings A wide range of interventions had a positive impact on indicators related to height, weight, wasting, and low birth weight. There were a total of 10 different outcome indica-tors for the four main anthropometric outcomes. A little more than half of the evaluations addressing a height-related indicator found program impacts on at least one group of children, and this was true for about the same share of interventions aimed at improving weight-related and wasting (low weight for height)-related indicators. About three-quarters of the 11 evaluations of interventions that aimed at improving birth weight indicators registered an impact in at least one specification, including five out of seven micronutrient interven-tions. There was no clear pattern of impacts across interventions in every intervention group there were examples of programs that did and did not have an impact on a given indicator, and with varying magnitude. Evaluations of the nutritional impact of programs supported by the World Bank, which are generally large-scale, complex, and implemented in low-capacity settings, show equally variable results. Even controlling for the specific outcome indicator, studies often targeted children of different age groups that might be more or less susceptible to the interventions. It is thus difficult to point to inter-ventions that are systematically more effective than others in reducing malnutrition across diverse set-tings and age groups. Differences in local context, variation in the age of the children studied, the length of exposure to the intervention, and differing methodologies of the studies account for much of the variability in results. Context includes factors like the level and local determinants of malnutrition, differences in the characteristics of beneficiaries (including their age), the availability of service infrastructure, and the implementation capacity of government. Outside of a research setting in the context of a large government program there are many things that can go wrong in either service delivery or the demand response that can compromise impact. Beyond this, there are social factors like the status of women or the presence of civil unrest that can affect outcomes. These findings underscore the conclusion that it should not be assumed that an intervention found effective in a randomized controlled trial in a re-search setting will have the same effects when im-plemented under field conditions in a different set-ting. They also point to the need to understand the prevailing underlying causes of malnutrition in a given setting and the age groups most likely to benefit in selecting an intervention. Further, impact evaluations need to supplement data measuring impact with data on service delivery and demand-side behavioral outcomes to demonstrate the plausibility of the findings, to understand what part of a program works, and to address weak links in the results chain to improve performance. There is scant evidence on the distribution of nutrition impacts who is benefiting and who is not or on the cost-effectiveness of interventions Just because malnutrition is more common among the poor does not mean that they will disproportio-nately benefit from an intervention, particularly if acting on new knowledge or different incentives relies on access to education or quality services. Only a third of the 46 evaluations looked at the distribution of impacts by gender, mother s education, poverty status, or availability of complementary health services. Only nine assessed the impacts on nutritional outcomes of the poor compared with the non-poor. Among the evaluations that did examine variation in results, several found that the children of more educated mothers or in better-off communities are be-nefitting the most. Bank-supported cash transfers, community nutrition, and early child development programs in six of eight countries had some impact on child anthropometric outcomes. Of the 12 impact evaluations of Bank support, all but one were of large-scale government programs with multiple interventions and a long results chain. Three-quarters found a positive impact on anthro-pometric outcomes of children in at least one age group, although the magnitude was in some cases not large or applied to a narrow age group. Most of the impact evaluations involved assessment of completely new programs and involved World Bank researchers. Most used quasi-experimental evaluation designs and two-thirds assessed impact after at most 3 years of program implementation. Only half of the evaluations documented the distribution of impacts and only a third presented information on the costs of the intervention (falling short of cost-effectiveness analysis). In two of the countries (Colombia and the Philippines) the evaluations likely had an impact on government policy or programs. Lessons A number of lessons for development practi-tioners and evaluators arose from the review of impact evaluations of World Bank nutrition support. For task managers: Impact evaluations of interventions that are clearly beyond the means of the government to sustain are of limited relevance. The complexity, costs, and fiscal sustainability of the intervention should figure into the decision as to whether an impact evaluation is warranted. Impact evaluations are often launched for the purpose of evaluating completely new pro-grams, but they may be equally or even more useful in improving the effectiveness of ongo-ing programs. There are methods for obtaining reliable impact evaluation results when randomized assignment of interventions is not possible for political, ethical, or practical reasons. For evaluators: In light of the challenges of evaluating large-scale programs with a long results chain, it is well worth the effort to assess the risks to disruption of the impact evaluation ahead of time and identify mitigation measures. The design and analysis of nutrition impact evaluations need to take into account the likely sensitivity of children of different ages to the intervention. For the purposes of correctly gauging im-pact, it is important to know exactly when delivery of an intervention took place in the field (as opposed to the official start of the program). Evaluations need to be designed to provide evidence for timely decision-making, but with sufficient elapsed time for a plausible impact to have occurred. The relevance of impact evaluations for po-licymakers would be greatly enhanced if im-pact evaluations were to document both the

Over the past decade the development community has focused more on measuring results, so the use of impact evaluations has expanded rapidly. IEG examines the relevance, quality, and influence of World Bank and IFC impact evaluations.

Nutrition investments affect human capital formation, which in turn affects economic growth. Malnutrition is intrinsically connected to human capital—undernutrition contributes to nearly half of child mortality, and stunting reduces productivity and earnings in adulthood. Improving nutrition requires a multisectoral effort, but it is difficult to identify and quantify the basic financing parameters as used in traditional sectors. What is being spent and by whom and on what? To address these questions, nutrition public expenditure reviews (NPERs) determine the level of a country’s overall nutrition public spending and assess whether its expenditure profile will enable the country to realize its nutrition goals and objectives. When done well, NPERs go beyond simply quantifying how much is spent on nutrition; they measure how well money is being spent to achieve nutrition outcomes and identify specific recommendations for improvement. A Guiding Framework for Nutrition Public Expenditure Reviews presents the key elements of an NPER and offers guidance, practical steps, and examples for carrying out an NPER. The book draws upon good practices from past NPERs as well as common practices and expertise from public expenditure reviews in other sectors. This handbook is intended for practitioners who are tasked with carrying out NPERs. Other target audiences include country nutrition policy makers, development partner officials, government technical staff, and nutrition advocates. The book presents data and analytical challenges faced by previous NPER teams and lays out the kinds of analyses that past NPERs have been able to carry out and those that they were unable to perform because of data or capacity constraints. It concludes with further work needed at the global and country levels to create the conditions necessary to conduct more comprehensive NPERs.

This year's report summarizes the short- and long-term impact of food prices on several MDGs, explores future trends, and review policy responses, from domestic policies such as social safety nets, nutritional programs and agricultural policies, to regional trade policies to support by the international community.

A chain is only as strong as its weakest link. When the links in the chain represent development projects, if individual projects fail to achieve their purpose, the development program's effectiveness is compromised. When the chain's links are strong and well-connected, the results are improved for the sector, country, and region. The role of multilateral banks is crucial; they inform the impact of development operations and support policymakers in decision making. This volume emphasizes that some crucial links in development tend to be systematically overlooked. In these matters, preoccupation with the immediate exigencies seems to come at the expense of attention to enduring problems-at a great cost to society. Development practitioners should apply policies that have produced results over time, ensuring that the links in the chain are strong, but too often they overlook those links-because of myopia, complexity, tradition, or special interests. This book will help policy makers and practitioners focus on the links that measure progress, apply lessons, and matter for lasting results.

The second edition of the Impact Evaluation in Practice handbook is a comprehensive and accessible introduction to impact evaluation for policy makers and development practitioners. First published in 2011, it has been used widely across the development and academic communities. The book incorporates real-world examples to present practical guidelines for designing and implementing impact evaluations. Readers will gain an understanding of impact evaluations and the best ways to use them to design evidence-based policies and programs. The updated version covers the newest techniques for evaluating programs and includes state-of-the-art implementation advice, as well as an expanded set of examples and case studies that draw on recent development challenges. It also includes new material on research ethics and partnerships to conduct impact evaluation. The handbook is divided into four sections: Part One discusses what to evaluate and why; Part Two presents the main impact evaluation methods; Part Three addresses how to manage impact evaluations; Part Four reviews impact evaluation sampling and data collection. Case studies illustrate different applications of impact evaluations. The book links to complementary instructional material available online, including an applied case as well as questions and answers. The updated second edition will be a valuable resource for the international development community, universities, and policy makers looking to build better evidence around what works in development.

Sub-Saharan Africa's natural resource-rich countries have poor human development. Children in these countries are more likely to die before their first birthday, more likely to be stunted, and less likely to attend school than children in other countries with similar income. Despite the current price downturn, extractives will remain an important part of Sub-Saharan Africa's growth story—using resource rents wisely remains a long term challenge. Governments must choose how to allocate resource rents between spending, investing in human or physical capital, or investing in global financial assets. The return to investing in physical and human capital will be high in countries where the capital stock is low. Moreover, higher levels of human capital make investments in physical capital more productive, which suggests that the optimal portfolio will involve investing in both. Human capital should be prioritized in many of Sub-Saharan Africa’s resource-rich countries because of the low starting point. Investing effectively in human capital is hard because it involves delivering services, which means coordinating a large number of actors and activities. Three dimensions of governance are key: institutions, incentives and information. Decentralization and leveraging the private sector are entry points to reforming institutional structures. Revenues from natural resources can fund financial incentives to strengthen performance or demand. Producing information, making it available, and increasing social accountability helps citizens understand their rights and hold governments and providers accountable. Improving the quality of education and health services is central to improving human capital. Two additional areas are promising. First, early child development—mother and newborn health, and early child nutrition, care, and education—improves outcomes in childhood and later on. Second, cash transfers—either conditional or unconditional—reduce poverty, increase household investments in child education, nutrition, and health, and increase the investment in productive assets which foster further income generation.