Skip to Content

UNU-WIDER Logo

30 Years of economics for development

Experimental and Non-Experimental Methods to Study Government Performance: Contributions and Limits

Project name/title
Experimental and Non-Experimental Methods to Study Government Performance: Contributions and Limits
Year:
2012
Abstract:
In recent years, field experiments using randomized trials have gained increasing popularity in the field of development economics. In particular, scholars have argued strongly for their use as the best means of identifying ‘what works’ in foreign aid. Field experiments have also become increasingly popular in political science, particularly in the study of democracy and elections. The project brings together economists and political scientists to investigate the use of field experiments in the study of governance, and their limitations with regards to ‘what works’, ‘what is scalable’, and ‘what is transferable’ in foreign aid. The project focuses on interventions aimed to enhance government performance in the areas of public service delivery, and contributes to the literature in three important ways: First, the project speaks to one of the core questions for development practitioners today – how donors can best help developing countries to improve their governance – and systematically explores the insights that can be gained on this question from randomised experimental design. While experimental methods have been increasingly used in the study of other areas of development assistance over the last decade, field experiments have covered only a limited range of issues related to governance. The project will build on the literature to provide an overview of key issues, findings, and challenges relevant to future work on governance. Second, the project speaks to one of the core criticisms of experimental work in political science: that it cannot and does not address ‘big’ questions and ‘big’ theories, much less help us to evaluate the national-level policy interventions that these theories suggest. The project explores to what extent such criticism is valid and whether in fact experiments can be designed to address some of the ‘big’ theories about governance. Third, in focusing on ‘what is scalable’ and ‘what is transferable’ in foreign aid, the project directly studies one of the core criticisms of experimental work more generally: that its findings are not externally valid and tell us little about ‘what works’ beyond the narrow context of the experiment. Furthermore, field experiments are contested by their short-term window of analysis, a fact that strongly limits the examination of social phenomena that occur in the course of years or even generations. Many scholars criticize experimental work, along these lines, but few studies in the social sciences have empirically investigated these criticisms. Project contributors employ use new and existing data to analyse precisely (1) what is gained and lost through the use of randomized controlled trials versus other approaches (e.g., quasi-experimental designs, survey techniques) and (2) the possibilities of employing new techniques in the design and analysis of randomized experimental trials that can improve their ability to speak beyond the narrow context of the experiment.
Keywords:
Field experiments, non-experimental methods, governance, government performance, public goods, social sectors, service delivery, developing countries, poverty, development economics, politics
Contact:

UNU-WIDER focal point:
Rachel M. Gisselquist, Research Fellow

Miguel Niño-Zarazúa, Research Fellow

Project Assistant:
Janis Vehmaan-Kreula

Project Meetings
Publications:
Content:

 

Quantitative Analysis in Social Sciences: A Brief Introduction for non-Economists
Miguel Niño-Zarazúa


Further Reading 

Identifying 'what works' in foreign aid: Experimental and non-experimental approaches
WIDER Angle September 2013

UNU-WIDER's ReCom programme is centred around four core questions: what works, what could work, what is scalable, and what is transferrable in foreign aid? One of the first challenges in addressing these questions is a methodological one: what is the best way to identify the policies that 'work'? This question has obvious links to another major debate in development studies over the use of experimental methods, and in particular, of randomized controlled trials (RCTs). While some researchers have argued that RCTs are the best means of determining what works in development policy (e.g., Banerjee 2007; Druckman et al. 2006), others raise a number of strong concerns (see, e.g., Deaton 2009; Ravallion 2009). 

What Can Experiments Tell Us About How to Improve Governance?
WIDER Angle October 2012 

Over the past decade, randomized controlled trials (RCTs) have become a staple of research in development economics. Proponents of RCTs have advocated for their use as the best means of identifying ‘what works’ in development, while sceptics voice strong concerns about their growing hegemony in the field. Last year, two influential books, Karlan and Appel’s More Than Good Intentions, and Banerjee and Duflo’s Poor Economics, summarized what RCTs can tell us about how to reduce global poverty. Sceptics such as Angus Deaton and Martin Ravallion point out that RCTs, even if well designed, are not the ‘gold standard’ to policy evaluation as they often rely on small samples (and small pilot interventions) that cannot tell us much about whether a policy would work if scaled up at national level, or transferred to different socioeconomic and political conditions.



Back to Top

^ Back to top

1995-2014 United Nations University World Institute for Development Economics Research

© CC BY-NC-SA 3.0 IGODisclaimer | Terms of Use
UNU-WIDER, Katajanokanlaituri 6 B, FI-00160 Helsinki, Finland
Tel: +358(0)9 6159911 | Fax: +358(0)9 61599333
mail: wider@wider.unu.edu/firstname.lastname@wider.unu.edu