Ethiopia has launched a pro-poor health services extension programme since 2003 to deliver preventive and basic curative health services to its inhabitants. Despite the massive support and recognition the programme has received, there has not been proper evaluation of its impact. This study has applied propensity score matching and regression adjustment techniques to evaluate the short-term and intermediate-term impacts of the programme on child and maternal health indicators in the programme villages. Empirical data for the study were collected from 3095 households from both programme and non-programme villages in rural Ethiopia. The estimated results indicate that the programme has significantly increased the proportion of children fully and individually vaccinated against tuberculosis, polio, diphtheria–pertussis–tetanus, and measles. The study finds heterogeneity in childhood immunisation coverage as a result of differences in terms of the number of health extension workers, in the quality of health posts and in terms of the educational achievement of mothers across programme villages. The proportions of children and women using insecticide-treated bednets for malaria protection are significantly larger in programme villages than in non-programme villages. The effect on preventive maternal care is rather limited. Whereas women in the programme villages appeared to make their first contact with a skilled health service provider significantly earlier during pregnancy, very little effect is detected on other prenatal and postnatal care services. Moreover, the programme has not reduced the incidence and duration of diarrhoea and cough diseases among under-five children.

Along with the other panellists in a session of this conference, the author was asked to discuss evaluation designs for three stylised interventions: conditional cash transfers, a transport sector programme, and an anti-corruption commission. This paper records his responses, and elaborates a little on some points, including references to the literature. He begins with some general suggestions on the issues to think about at the outset of any evaluation. He then tries to illustrate these points with reference to the three stylised interventions.

In an effort to increase access to effective anti-malaria drugs to the rural poor, the Kenyan government has partnered with a local non-governmental organisation to distribute the drugs free of charge using a microfranchise system in small privately-owned rural shops. This study uses difference-in-difference to evaluate the effectiveness of the programme in increasing access to the drugs and hence on its impact on malaria morbidity and mortality. If effective, this system can be recommended for adoption in the distribution of other essential medicines to help in achieving some of the health-related millennium development goals (MDGs) in Africa and Asia. The results show that the programme has no significant impact on malaria morbidity in treatment areas, but there is an impact once treatment is defined with reference to distance to facility. The impact is however less when patients have to walk longer distances to access drugs. In addition, the programme is found to have significant spillover effects to neighbouring untreated areas. Programme impact on mortality is generally insignificant. The programme is therefore recommendable for replication with particular emphasis on putting measures to reduce distance to facility by establishing more outlets nearer to the patients.

The authors were asked to discuss specific methodological approaches to evaluating three hypothetical interventions. This article uses this forum to discuss three misperceptions about randomised trials. First, nobody argues that randomised trials are appropriate in all settings, and for all questions. Everyone agrees that asking the right question is the highest priority. Second, the decision about what to measure and how to measure it, that is, through qualitative or participatory methods versus quantitative survey or administrative data methods, is independent of the decision about whether to conduct a randomised trial. Third, randomised trials can be used to evaluate complex and dynamic processes, not just simple and static interventions. Evaluators should aim to answer the most important questions for future decisions, and to do so as reliably as possible. Reliability is improved with randomised trials, when feasible, and with attention to underlying theory and tests of why interventions work or fail so that lessons can be transferred as best as possible to other settings.

This paper is the protocol for a synthetic review of microfinance. The protocol describes the objectives and scope of the review, the search strategy, inclusion criteria, coding, and synthesis options.

Very few of the (many) aid-financed rural road projects in developing countries have been the subject of rigorous impact evaluations. Assessing the welfare impacts of rural roads poses a number of problems, with implications for data collection and evaluation methods. This paper surveys the problems and discusses some practical implementation issues related specifically to conducting an impact evaluation of a rural roads project that is assigned to some geographic areas but not to others.

Project monitoring, impact evaluation, results-based management and effects assessment are all based on the comparison of observed changes in the project population with the situation prior to the launch of the project. However, a significant proportion of each of these types of estimation has to be made without access to baseline data on the project or control groups. This paper examines some of the reasons for this widespread lack of baseline data and describes and illustrates a range of techniques that can be used for the ‘reconstruction’ of baseline data.