Skip to main content
SpringerLink
Log in

Current Challenges in Impact Evaluation

  • Special Debate Section
  • Published:
The European Journal of Development Research Aims and scope Submit manuscript

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3

Notes

  1. But not ironically on the NGO, which implemented the pilot study used for the evaluation; see Roetman (2011).

  2. Specifically, studies were excluded if the treatment group were concurrently treated for schistosomiasis unless the control group was also being treated for schistosomiasis.

  3. Such a review is currently underway; see the protocol of Welch et al (2014).

  4. It is also worth noting that the intervention was conducted in NGO schools. In government schools, unions would likely resist both the monitoring and the change in their terms and conditions, and thus there is limited external validity to the study.

  5. The exchange occurred during a presentation at the Public Health Foundation of India, New Delhi; personal communication from Radhika Menon.

  6. And doing so raises many intellectual challenges as I found when applying this approach in studies on education (World Bank, 2004, 2005), health (World Bank, 2006) and rural electrification (World Bank, 2007). For example, there is no consensus, nor even much literature, on valuing fertility reduction.

References

  • Abolghasemi, H., Hosseini-Divkalayi, N.S. and Seighali, F. (2010) Blood donor incentives: A step forward or backward. Asian Journal of Transfusion Science 4 (1): 9–13.

    Article  Google Scholar 

  • Ashraf, N., Bandiera, O. and Jack, K. (2013) No Margin, No Mission? Evaluating the Role of Incentives in the Distribution of Public Goods, 3ie Impact Evaluation Report No. X [New Delhi: International Initiative for Impact Evaluation].

  • Bond, T., Kimenyi, M., Mwabu, G., Ng’ang’a, A. and Sandefu, J. (2013) Scaling Up What Works: Experimental Evidence on External Validity CGD Working Paper 321. [Washington DC: Centre for Global Development].

  • Cardenas, J.C. and Carpenter, J. (2008) Behavioural development economics: Lessons from field labs in the developing world. Journal of Development Studies 44 (3): 311–338.

    Article  Google Scholar 

  • Dhaliwal, I., Duflo, E., Glennerster, R. and Tulloch, C. (2012) Comparative cost-effectiveness analysis to inform policy in developing countries: A general framework with applications for education (mimeo) http://www.povertyactionlab.org/publication/cost-effectiveness, accessed 15 August 2013.

  • Duflo, E., Hanna, R. and Ryan, S.P. (2010) Incentives work: Getting teachers to come to school (mimeo) http://economics.mit.edu/files/5995, accessed 15 August 2013.

  • Easterly, W. (2007) The White Man’s Burden: Why the West’s Efforts to Aid the Rest Have Done So Much Ill and So Little Good. Oxford: Oxford University Press.

    Google Scholar 

  • Goldacre, B. (2012) Bad Pharma: How Drug Companies Mislead Doctors and Harm Patients. London: Harper Collins.

    Google Scholar 

  • Guijt, I. and Roche, C. (2014) Does impact evaluation in development matter? Well it depends what it’s for! European Journal of Development Research 26 (1): 46–54.

    Article  Google Scholar 

  • Harrison, G.W. (2014) Impact evaluation and welfare evaluation. European Journal of Development Research 26 (1): 39–45.

    Article  Google Scholar 

  • Hotez, P.J., Mistry, N. and Fenwick, A. (2012) Deworming should remain an essential cornerstone for NTD control, PLOS Blogs 18 July, http://blogs.plos.org/speakingofmedicine/2012/07/18/should-deworming-policies-in-the-developing-world-be-reconsidered/, accessed 16 August 2013.

  • Humphreys, M., Sanchez de la Sierra, R. and van der Windt, P. (2013) Aid and Accountability: Governance Effects of a Community-driven Program in Eastern Congo. 3ie Impact Evaluation Report No. X [New Delhi: International Initiative for Impact Evaluation].

  • Hunter, J.E. and Schmidt, F.L. (2004) Methods of Meta-Analysis: Correcting Error and Bias in Research Findings, 2nd edn. London: Sage Publications.

    Book  Google Scholar 

  • Karlan, D. and Appel, J. (2011) More Than Good Intentions: Improving the Ways the World’s Poor Borrow, Save, Farm, Learn, and Stay Healthy. New York: Penguin.

    Google Scholar 

  • Lensink, R. (2014) What can we learn from impact evaluations? European Journal of Development Research 26 (1): 12–17.

    Article  Google Scholar 

  • Martinez, S., Naudeau, S. and Pereira, V. (2013) The Promise of Preschool in Africa: A Randomized Impact Evaluation of Early Childhood Development in Rural Mozambique. 3ie Impact Evaluation Report No. X [New Delhi: International Initiative for Impact Evaluation].

  • Menon, R. and de Hoop, T. (2013) Of rumours related to blood, poison and researchers: A pitch for an interdisciplinary approach to impact evaluation, http://www.3ieimpact.org/en/blog/2013/01/28/rumours-related-blood-poison-and-researchers/, accessed 15 August 2013.

  • Miguel, E. and Kremer, M. (2004) Worms: Identifying impacts on education and health in the presence of treatment externalities. Econometrica 72 (1): 159–217.

    Article  Google Scholar 

  • Mukherjee, C., Wuyts, M. and White, H. (1997) Econometrics and Data Analysis for Developing Countries. London: Routledge.

    Google Scholar 

  • Nichter, M. (1985) Drink boiled water: A cultural analysis of a health education message. Social Science and Medicine 21 (6): 667–669.

    Article  Google Scholar 

  • Null, C., Kremer, M., Miguel, E., Hombrados, J.G., Meeks, R. and Zwane, A.P. (2012) Willingness to pay for cleaner water in less developed countries: Systematic review of experimental evidence [New Delhi: 3ie] http://www.3ieimpact.org/media/filer/2012/05/28/sr006.pdf, accessed 15 August 2013.

  • Picciotto, R. (2014) Is impact evaluation evaluation? European Journal of Development Research 26 (1): 31–38.

    Article  Google Scholar 

  • Rankin, N., Levinsohn, J., Ira, J., Harris, N. and Faulkner, D. (2013) A Youth Wage Subsidy Experiment for South Africa. 3ie Impact Evaluation Report No. X [New Delhi: International Initiative for Impact Evaluation].

  • Roetman, E. (2011) A can of worms? Implications of rigorous impact evaluations for development agencies [New Delhi: International Initiative for Impact Evaluation] http://www.3ieimpact.org/en/evaluation/working-papers/working-paper-11/, accessed 15 August 2013.

  • Rogers, E.M. (2003) Diffusion of Innovations. New York: Free Press.

    Google Scholar 

  • Savedoff, W. (2013) ‘Impact Evaluation?: Where have we been? Where are we going?’ Presentation to impact evaluation: How can we learn more? Better? CGD. Washington DC, 17 July, http://international.cgdev.org/sites/default/files/Savedoff.pdf, accessed 15 August 2013.

  • Thaler, R. and Sunstein, C. (2008) Nudge: Improving Decisions about Health, Wealth, and Happiness. London: Penguin Books.

    Google Scholar 

  • Taylor-Robinson, D.C., Jones, A.P. and Garner, P. (2007) Deworming drugs for treating soil-transmitted intestinal worms in children: Effects on growth and school performance. Cochrane Database of Systematic Reviews, Issue 4. Art. No.: CD000371. doi: 10.1002/14651858.CD000371.pub3.

  • Taylor-Robinson, D.C., Maayan, N., Soares-Weiser, K., Donegan, S. and Garner, P (2012) Deworming drugs for treating soil-transmitted intestinal worms in children: Effects on nutrition and school performance, http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD000371.pub5/full#CD000371-bbs2-0023, accessed 15 August 2013.

  • Titmuss, R. (1970) The Gift Relationship: From Human Blood to Social Policy. London: Pelican Books.

    Google Scholar 

  • Vajja, A. and White, H. (2008) Can the World Bank build social capital? The experience of social funds in Malawi and Zambia. Journal of Development Studies 44 (8): 1145–1168.

    Article  Google Scholar 

  • Waddington, H., Snilstveit, B., Garcia-Hombrados, J., Vojtkova, M., White, H. and Anderson, J. (2014) The impact of farmer field schools: A systematic review, Campbell Library.

  • Welch, V. et al. (2014) Deworming and adjuvant interventions for improving the developmental health and well-being of children in low- and middle-income countries: A systematic review and network meta-analysis.

  • White, H. (2003) Social organisation, civic responsibility and collective action: Game theory models of community participation. Oxford Development Studies 31 (2): 149–158.

    Article  Google Scholar 

  • White, H. (2009) Theory-based impact evaluation: Principles and practice. Journal of Development Effectiveness 1 (3): 271–284.

    Article  Google Scholar 

  • White, H. (2013) The use of mixed methods in randomized control trials. New Directions for Evaluation 138: 61–73.

    Article  Google Scholar 

  • White, H. and Masset, E. (2006) The Bangladesh integrated nutrition program: Findings from an impact evaluation. Journal of International Development 19: 627–652.

    Article  Google Scholar 

  • World Bank (2002) Social Funds: An Assessment. Washington DC: OED, World Bank.

  • World Bank (2004) Books, Buildings and Learning Outcomes: An Impact Evaluation of World Bank Support to Basic Education in Ghana. (assisted by Edoardo Masset) Washington DC: OED, World Bank.

  • World Bank (2005) Maintaining Momentum to 2015? An Impact Evaluation of Interventions to Improve Maternal and Child Health and Nutrition in Bangladesh. Washington DC: OED, World Bank.

  • World Bank (2006) An Impact Evaluation of the Andhra Pradesh Second and Third Irrigation Projects: Poverty Reduction with Limited Economic Benefits. Washington DC: IEG, World Bank.

  • World Bank (2007) The Welfare Impact of Rural Electrification: A Reassessment of the Costs and Benefits. Washington DC: IEG, World Bank.

  • Ziliak, S.T. and McCloskey, D.N. (2008) The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives. Ann Arbor, MI: University of Michigan Press.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. International Initiative for Impact Evaluation (3ie), New Delhi, India

    Howard White

Authors
  1. Howard White
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

White, H. Current Challenges in Impact Evaluation. Eur J Dev Res 26, 18–30 (2014). https://doi.org/10.1057/ejdr.2013.45

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1057/ejdr.2013.45

Search

Navigation