For the past two years, I have spent a significant share of my time working on energy access in India. This is quite a departure from my earlier approach, which was heavy on game theory and analysis of data at the national level.
As I initiated projects in India, my natural instinct was to go for the currently trendy method of field experiments. In a field experiment, the effect of an intervention on some outcome of interest is tested. The effect can be measured because the intervention is randomly assigned to the treatment group, while the remainder of the study sample are the control group for comparisons.
The experimental method is well-suited for the analysis of causal effects. In my research, I am evaluating the effects of distributed solar electricity on rural livelihoods (Mera Gao Power), marketing campaigns on sales of solar home systems (Boond), and solar lighting on street vendor business (Nidan). While the projects have been more than a little challenging (yes, I was warned in advance), it’s been a great learning experience and lots of fun.
The main downside of the experimental method is rigidity. If the goal is to test an existing intervention rigorously, then a field experiment is a good approach. However, the method is too cumbersome for piloting, developing, and modifying new approaches. This is a severe limitation in the case of distributed energy generation, where new ideas are badly needed.
For example, some of my field experiments clearly suggest that the current approach suffers from deficiencies. Grant conditions, pre-analysis plans, and the logic of publication create professional incentives to continue with the experiment and report cleanly identified null results. At this stage, an engineer would have stopped the experiment and tried something else.
In the future, social scientists need to expand their methodological portfolio and create a publishing culture that values studies that focus on innovating new approaches. Currently, the leading social science journals do not publish this type of research. We have modelled our enterprise after medical research, as opposed to engineering. I suspect methodological sophistication is driving this trend at the expense of substance. We need a wider range of methods and a more diverse set of outcomes and results that are publishable.