Making Impact Evaluation Matter

24 09 2014

Earlier this month I had the pleasure of attending the Making Impact Evaluation Matter conference in Manila, hosted by the Asian Development Bank (ADB), 3ie, and the Philippine Institute for Development Studies (PIDS). During the conference, I took part in a WASH Impact Evaluation Design Clinic and gave a presentation on the results of the impact evaluation I directed of the MCC-funded Rural Water Supply Program in Nampula, Mozambique. Whereas the final impact evaluation report was based on all of the data we collected, the results I presented in Manila focused only on the panel data obtained from the baseline (2011) and follow-up (2013) studies. The results from the panel data analysis (shown in the presentation below) align well with those developed from the full data set.


During the conference, Eric Vance spoke about his LISA2020 vision to create 20 statistical consulting laboratories in 20 developing countries by the year 2020. Eric is the Director VT’s Laboratory for Interdisciplinary Statistical Analysis (LISA) that was a key partner in the MCC impact evaluation. For more information on LISA2020 visit the program website.

VT doctoral student Marcos Carzolio also attended the conference and discussed his impact as the field statistician on the quality of data collected during the 2013 follow-up study. Marcos’s talk on Leveraging the Field Statistician to Ensure High Quality Impact Evaluations received the award for “Second Place Best Presentation by a Young Researcher.”

The slideshow below presents a number of images and tweets (#IEmatters) from the conference.

This slideshow requires JavaScript.

Update from the Field

1 06 2013

I started writing this post a week ago while flying from Maputo to Nampula in Mozambique, but the usual challenges of setting up a large-scale research project delayed my intentions. One challenge is finding/installing a good Internet connection, which we overcame by purchasing a number of Movitel USB modems that are working surprisingly well.

I’m in Nampula with a team of researchers from Virginia Tech (Eric Vance, Emily Van Houweling, and Marcos Carzolio), Stanford University (Jenna Davis and Kory Russel), and WE Consult (our in-country partner) to undertake a follow-up study for an evaluation of the Millennium Challenge Corporation’s rural water program in the province. We undertook the baseline study in 2011 and plan to complete the follow-up study over the next two months.

Over the past week, we have begun our enumerator training that has focused primarily on reviewing each module of the household survey (which consists of more than 600 questions that will be navigated using logic) and practicing skills such as how to measure a child and use GPS devices (that are needed to find the same households we surveyed in 2011). Next week we will continue to refine the various surveying instruments with the enumerators (who are mostly local to Nampula) and begin practicing how to take water samples, among other activities. We will soon select our team leaders from the twenty enumerators we are training, who will each receive specific training on the various research protocols we will be using.

We have established a base camp in a house in the city of Nampula, which is proving to be a pleasant place to work/live, even if there is no water most days. There are several major construction projects happening in the city that have severely constrained the water supply that only comes once a day for several hours if we are lucky.

I have included some pictures below of the activities so far. I will post some more following the pilot study that is planned for later next week.

This slideshow requires JavaScript.