ELLA Comparative Research: One Year In


Author: Don Leonard

One year ago, twelve Latin American and African centres came together in Lima to advance a research agenda founded on two basic propositions. First, that comparing the experiences of Latin American and African countries can generate new insights on difficult and important questions. Second, that in order for these insights to be valuable to policymakers and stakeholders this process should be guided by scientific rigour.

Over this year the teams have grappled with a challenge at the heart of this phase of ELLA. The teams were not assembled based on any established logic of scientific comparison like Mill’s method of agreement or difference. Rather, they were selected based on the area expertise of the researchers, and by their mutual interest in tackling a common social problem. Can development experiences in Latin American really be compared to African experiences under these conditions? Can those experiences form the basis for better public policies? In short, what are we comparing, and why?

Not surprisingly, we have learned that cross-regional comparison is hard. The research centres have worked diligently to overcome the mundane obstacles to conducting joint projects with fellow researchers based half a world away. Meeting these practical challenges has been crucial as the research centres move forward on a design that shares a common analytical framework. Within the topics that originally brought the six teams together—smallholder livelihoods, informal labour markets, implementation of domestic violence laws, community-based crime prevention, local content requirements in resource extraction, and horizontal accountability of the executive branch—each of these topics contained a seemingly infinite number of central research questions and outcomes to be explained. What aspects of smallholder livelihoods are most pressing in Kenya’s policy environment? Are those issues also present and important in the Andean context? As Mexico and Ghana set about implementing different domestic violence laws under different conditions (like matrilineal versus patrilineal systems of property rights, for example), are there shared challenges that can be usefully explored in tandem? The centres have been equally diligent in their efforts to design research projects with a scope that is both focused and policy relevant.

As the teams began to address these issues and converge on central research questions, they were confronted by a vast array of potential explanatory factors. Could Latin American and African researchers maintain a common analytical framework while still remaining true to the unique characteristics of their policy environments? Can the varying realities of Colombian and South African labour markets—differing levels of unemployment and differing approaches to labour market regulation—generate common insights about the relationship between informal labour and inclusive growth? Can an analytical framework focusing on capacity and incentives explain executive branch accountability behaviour equally well in both Kenya and Argentina?

Over the last year, the teams have met twice to consider these questions and to set about putting their research designs into motion. In some cases the central questions and analytical frameworks have remained relatively intact, allowing the teams to focus on selecting the correct methods of analysis and cope with the different kinds of data availability on either side of the Atlantic. In other cases the central questions themselves evolved as the teams delved deeper into their topics. What if specific historical and cultural forces shaping communal land use, rather than government regulations, are the main drivers determining smallholder livelihood in Kenya and Peru today?

What has emerged after a busy year of collaboration and comparison between research centres is an acute awareness of the challenges that this type of South-South learning entails. Looking ahead, there will be a new set of challenges associated with extracting the kinds of comparative lessons that can transform how different societies address common challenges. Problems will emerge as the analytic frameworks developed in the confines of the ELLA workshops encounter the realities of the policy environment. These “problems” should be viewed as opportunities to capture new and valuable information. But for this unique kind of comparative effort to succeed, it will be essential for the research centres to come together as a team to make sense of this information, adapting their comparative frameworks accordingly.

I conclude with some brief reflections on how this process of adaptation might unfold with scientific rigour. If the broader project happens to include cases in both regions that are similar in outcomes or pre-conditions, a straightforward comparative analysis can of course be made. If the cases do not conform, there are at least three possibilities. One is to use alternative statistical or process-tracing techniques designed to cope with the inability to recreate experimental conditions. The research centres have each been briefed on these techniques during a four day design and methods workshop, and those who have chosen to employ them are benefiting greatly from additional external methodological support organised by ELLA.

A second possibility is that the research centres might identify “cases within the cases”, ones inviting more direct comparisons that students of Mill would recognize as being “most similar” or “most different.”

Finally, it is possible that fundamental differences in the Latin American and African contexts will produce comparative findings that simply do not lend themselves to straightforward analysis of the causal relationship between Xs and Ys. Here as well, though, policymaking processes can be advanced by applying the tools of science to the comparative effort. What are the confounding variables, contextual factors, or other scope conditions that prevent direct comparison? What can policymakers and stakeholders learn about the sensitivity of different policy prescriptions to the contexts in which they are being considered? Are local content requirements designed for one extractive economy transferrable to another? Is the effectiveness of community-based crime prevention programs sensitive to historical or cultural factors? If so, what are the implications of those contextual factors for the development of new crime prevention strategies?

When we consider the mixed record of international development paradigms under conditions where policy recommendations designed for one context are transplanted to another, it becomes clear that knowledge about non-generalizability is just as important. Whatever shape the final comparative effort takes, the degree of focus the ELLA teams have brought to their topics and the rigour with which they have set about evaluating the regional evidence bodes well for the kinds of contributions they will be able to make to their policy environments. For that reason alone, it is a most happy first anniversary indeed.