Understanding Sustainability Interventions: An Assessment of Experimental Evidence in the Coffee Sector | 25, Issue 20
Economists DAVIDE DEL PRETE and ROCCO MACCHIAVELLO recently completed a literature review of sustainability interventions for the Coffee Science Foundation; here, with PETER GIULIANO, they summarize its findings.
More than a decade ago, the sustainability community of the Specialty Coffee Association came up with a motto for their work: “Make Coffee Better.” It was a simple expression of their mission to improve the coffee industry in whatever way they could. The meaning of “better” was intentionally broad and therefore inclusive: to make coffee environmentally better, socially better, morally better, and economically better for everyone in the value chain. It wasn’t long before the motto caught on outside the sustainability community—you started to see #makecoffeebetter everywhere, and before long, it became the motto for the SCA itself.
The popularity of #makecoffeebetter as a rallying cry for specialty coffee is a great credit to our industry. Though it has come to encompass a dedication to quality as well as sustainability, it is clear evidence that ours is a community dedicated to improvement. It is hard to find a coffee company who is not committed to improving their community and the larger community of those who rely on coffee for their livelihoods. And, as transparency and global communications have put more coffee companies in communication with their value chains, the desire to improve has extended to the countries where coffee is grown, which includes people who experience rural poverty. So, for decades now, a hallmark of the specialty coffee community has been a dedication to supporting development initiatives aimed at making coffee producers’ lives better. I am proud to work in an industry where this instinct is real and persistent, and I respect the individuals and companies who have invested in development projects aimed at improving livelihoods in coffee.
However, in our enthusiasm to improve coffee for everyone, we sometimes fail to take the time to assess whether our efforts are working. It is not well known in the industry which kinds of intervention are most effective—many coffee companies and development institutions invest based on weak evidence of effectiveness. For this reason, the Coffee Science Foundation, in pursuit of the SCA’s sustainability agenda, sought to survey the academic literature. With the support of a generous grant from Keurig Dr Pepper, we were thrilled to partner with leading economists Davide Del Prete and Rocco Macchiavello to perform this review. The intention, in the words of KDP’s Director of Supply Chain Sustainability, Whitney Kakos, is to “provide data-backed insights that enable stakeholders in the coffee industry like KDP to optimize the impact of their investments on farmer livelihoods.” In other words, to make information available so we can all make coffee better. We’re pleased to offer this article summarizing the results of the review.
Peter Giuliano
Chief Research Officer, SCA
Almost everyone in coffee is aware that coffee is an essential livelihood for millions of coffee producers, but of the more than 12.5 million farmers worldwide who rely on coffee cultivation as their primary source of income, at least 5.5 million live below the international poverty line of US$3.20 a day.
Smallholders, who often farm one or two hectares, make up 95 percent of those living below the international poverty line, and face several challenges in improving production and productivity. Several interventions and programs—like extension services, fertilizer subsidies, and training—have been deployed in agriculture to improve farmer livelihoods. In the coffee sector, certification systems and buyer-driven sustainability programs have been a popular approach to address what many see as a critical issue for the coffee industry, but surprisingly, very few of these have been rigorously evaluated. Despite their importance, there has been a lack of a systematic review of programs and interventions in the coffee sector (to date). Compounding this, the evidence that does exist is not widely known by users of and advocates for these programs. The full paper we produced aims to fill this gap and provide a comprehensive assessment of the experimental and quasi-experimental studies (more on the difference between these two categories later) along the coffee value chain.
Questions of Productivity
Before we ask a “how” question (“how can coffee productivity be improved?”), we needed to first ask two other questions: Can productivity be improved? Should it be improved? Massive differences in productivity, both across countries and within countries, currently exist (see Figure 1).
Natural factors like geography, climate, and land quality explain some of this variation. However, recent research has shown that low agricultural productivity is not only due to unfavorable geographic endowments, but also to economic choices made by farmers.[2] This suggests that investing in good agricultural practices and yield-boosting technologies can improve agricultural productivity.
Further, several studies suggest that agricultural productivity investments may be the best strategy for poverty reduction. A study by de Jainvry and Saudolet showed that productivity growth where those living below the poverty line work in agricultural and rural areas was more effective in reducing poverty than structural transformation (i.e., an economic shift away from agriculture to other sectors).[3]
Having established that improvements in productivity are both possible and beneficial, we decided to review what does and doesn’t work when it comes to improving productivity. With the support of the Coffee Science Foundation, underwritten by a grant from Keurig Dr Pepper, we undertook a project to gather good-quality evidence on various interventions’ effects on productivity. Let’s begin by discussing what evidence we tried to gather.
Figure 1: Coffee productivity by country, from a study in progress conducted using Enveritas Data.[1] In this “box and whisker” graph, each country’s range of coffee yields by individual farm is shown here. The “whiskers” to each side of the box represent the minimum and maximum yields, and the sides of “the box” are the first (left) and third (right) quartiles, with the center line indicating the median production for each country.
Selecting the Studies
Several researchers have studied the coffee sector. However, not all studies are equally rigorous in their evaluation methods. This is an increasingly important topic in economic and development research, since observational studies often provide unclear results. As an example: you might want to know about whether a free mosquito net program reduces malaria, so you study a community where there was a free mosquito net program, but the malaria rate stays the same. You might assume from this (logically) that the program was ineffective, but what if the program kept the malaria in the region from getting worse? Without a counterfactual example, known as a “control” in scientific experiments, it’s difficult to accurately measure the impact of a given intervention. As in all good science, evidence requires a carefully controlled experiment to deliver useable information. This approach, which focuses on experimental evidence, can deliver much more reliable results than can traditional observational studies.
For this research project, we decided to select studies that have either an experimental or quasi-experimental research design. Experimental research designs leverage a technique called Randomized Control Trials (RCTs), considered to be the gold standard for impact evaluations. This is a type of experimental design where individuals or units are randomly assigned to “treatment” and “control groups,” which allows us to compare outcomes and more clearly assess impact. However, since these controlled experiments are not always possible, researchers often resort to the use of what’s known as a “quasi-experimental” design. This attempts to mimic the control/treatment group approach to estimate the causal impact of an intervention without randomly assigning individuals or units to each group, as it relies on observational data.
We were able to identify only eight studies in the sector that met our criteria of being experimental or quasi-experimental. Of the studies that were conducted, each one aimed to establish the effect (or the lack of effect) of different types of interventions, including information and training, cash transfers, voluntary sustainability standards (VSS), and bundled programs.
Intervention Insights
In evaluating these studies, we were able to glean some key insights from the research. Again, though some of these conclusions might be counterintuitive, they are supported by the highest-quality studies we were able to find and analyze. The insights are organized according to the specific intervention studied.
Agricultural Training
Information and communication are crucial to agriculture: timely and accurate information on good agricultural practices, planting strategies, inputs, and prices can help farmers maximize yields and profits, and cope with changes in weather and soil conditions or pest and disease epidemics.[4]
We reviewed two papers that study the role of information, particularly in the adoption of good agriculture practices in coffee cultivation, namely Abate et al. (2021) and Duflo et al. (2020).[5] In both studies, the authors collaborated with a private partner (TechnoServe) to deliver training sessions to farmers.
The training sessions led to increased adoption of stumping—a positive agricultural practice—in Ethiopia. In Rwanda, however, the training sessions did not lead to increased adoption of rates of various practices like pruning, mulching, and stumping. Further, the evidence from the 2020 study by Duflo et al. suggests that complex information (e.g., multifaceted agricultural practices) does not spread to farmers who are not directly included in the training programs. In the same study, the authors also highlight that information by itself would not lead to improved outcomes when farmers face other binding constraints such as resources and labor.
Direct Cash Transfers
When addressing resource constraints, direct cash transfers to recipients (rather than in-kind offerings or subsidies) have proved to be quite impactful. A 2016 Overseas Development Institute (ODI) report highlights that several studies have found positive impacts from cash transfers on agricultural investments towards inputs as well as the accumulation of agricultural productive assets.[6] Another study, an evaluation of a large cash transfer program (GiveDirectly, 2019), showed that while the program resulted in significant income gains for farmers, most of this increase was driven by non-coffee sources.[7] This suggests that the farmers’ coffee production was not suffering from resource constraints, but from low demand: there was not enough demand for coffee that incentivized farmers to invest in coffee. Additionally, this type of highly subsidized intervention is quite expensive and therefore may not be scalable or sustainable.
Certifications
Certification programs require farmers to adhere to certain quality, environmental, and sustainability standards that may lead to productivity improvements. The studies we reviewed assessed the impact of 4C certification in Indonesia and Fairtrade International’s Fair Trade certification in Costa Rica.[8]
First, these studies found that certification directly boosted coffee productivity only if coffee cultivation was already perceived to be a stable livelihood strategy worthy of investment (rather than as a resilience crop). However, even in these livelihood strategy contexts, if there is not enough demand for certified coffee—or if mills do not have the capacity or know-how to access buyers of certified coffee—then the gains of certification are reduced. These studies also found that implementing certification programs comes with significant operational costs that often erode the premiums that mills capture, resulting in meager trickledown to the producers.
Buyer-driven Programs
While certification programs force producer organizations to adhere to standards that may lead to productivity improvements, the premiums realized by these standards are often too low to drive transformative outcomes (as outlined above, this is mainly the result of high operational costs of compliance and demand uncertainty). Buyer-driven programs address these challenges by assuring demand and covering program implementation costs for mills.
One of the eight studies covered this kind of program—the “Sustainable Quality Program” provided by a large buyer in Colombia.[9] In addition to quality requirements found in existing certification programs, this buyer-driven program assured export premiums, farm-gate premiums, and a lump-sum contribution to cover program implementation costs. The authors found that the program resulted in a massive quality improvement, higher prices at farm gate, and land expansion (suggesting that the program was valuable to farmers). It is worth noting, however, that this approach has other trade-offs: buyers are not obligated to purchase all the coffee that meets their program standards (though, in this case, the buyer did).
Expanding the Scope
As there was limited research focused on coffee, we expanded our search to the broader agricultural sector to gain insights into transformation in other agricultural sectors. We reviewed 27 studies and 6 academic reviews (which each encompass multiple different studies), and found they broadly covered three different categories of interventions: those primarily addressing capital constraints and risk (i.e., interventions that address a lack of funding or resources in order to apply yield-enhancing agricultural technologies); those primarily addressing gaps in information (i.e., interventions that shared “best practice” knowledge); and those that bundled multiple types of interventions into a single program.
Capital Constraint and Risk Interventions
Yield-enhancing technologies such as agricultural practices and inputs like seeds, fertilizers, and pesticides could prove to be detrimental in boosting agricultural production for farmers. Despite the obvious benefits, the adoption of these technologies remains low in many developing country contexts. Various studies argue that one of the hindrances to adoption of yield-enhancing technologies is the lack of capital and the credit constraints for farmers.[10] Eight of the studies we reviewed in the expanded scope addressed interventions aimed at addressing these capital constraints, and we found four broad insights.
First, the practice of extending microcredit to farmers does not always have the same impact (although some studies did suggest it was a high-impact intervention). However, the number of participants who used the credit facilities offered as an intervention was quite low (less than 25 percent). It’s possible this was because the credit programs on offer (characterized by microloans and micro-repayments) weren’t quite adapted to the needs of the agriculture sector, which typically requires a “lumped” initial investment for inputs and results in the same lumped payouts once or twice a year, corresponding to the harvest seasons. This also confirmed the second insight across these studies, that the financial program on offer must be tailored to the agriculture cycle. In other words, loans need to be available when they’re likely to be needed, i.e., before inputs would ideally be planted (new stock) or applied (fertilizer, pesticides, etc.).
These studies also highlighted that offering capital and subsidies at scale is quite expensive. One study[11] showed that different farmers have different returns to capital; targeting or directing credit to farmers who are truly credit constrained could yield productivity gains and enable agricultural transformation. And finally, this group of studies suggested that—particularly for smaller farmers who have lower returns—investment, including credit or the adoption of technology, becomes generally risky, as it may not lead to commensurate gains. In such cases, this risk (that the farm size will limit any return on investments) is the binding constraint rather than the lack of credit. In these situations, interventions that alleviate risk, such as insurance or weather-resistant crop varieties, have been seen to be more effective.
Informational Interventions
A lack of awareness and correct information prevents farmers from adopting and harnessing the right tools and strategies to improve their agricultural productivity even when capital is available. We reviewed three analog and six digital information interventions and found several recurring themes in these studies.
First, choosing who has access to training or information is important: while it’s expensive to go in person and teach all the target farmers about a certain new technology, the expectation that information and technology will automatically diffuse from a randomly selected group of farmers in a region is faulty. Targeting the “right group” of farmers by analyzing social networks helps to address this, but social network mapping is expensive. Some of the studies we reviewed found that targeting information/training to farmers who represent the average farmer in the village (“peer farmers”) and then incentivizing the group of peers to spread the information can really boost diffusion. More visible and salient tools, like a plot of land where techniques can be demonstrated, can also be used as an inexpensive strategy to disperse information, particularly to farmers outside of mainstream social networks.
Digital technologies have their own constraints, too. Although mobile phones are increasingly available, access to mobile phones and the internet remains quite low in rural economies of the developing world. A common digital intervention is the provision of frequent price information to farmers, with the expectation that farmers could use this information to their advantage to earn higher profits. However, these kinds of informational interventions often shared information (i.e., price fluctuations) that was not new or that farmers already had another way of accessing. Even if they did provide new information, the studies also found that farmers often could not act upon it because of imperfectly competitive market structures. For example, high transport costs and fragmented markets meant that farmers could not sell their produce in the market with the highest price simply because it was too far away. In other cases, farmers did not have access to an alternative market, leaving them with no alternative but to sell at the price available.
Bundled Interventions
It’s clear that farmers face multiple constraints; facilitating agricultural transformation may require a more holistic approach. An increasing body of literature studies the effectiveness of bundled interventions that simultaneously address multiple constraints faced by farmers. We reviewed two such studies: one that eased liquidity constraints, present bias, and contractual risk by providing index insurance and contract farming; and another, which sought to ease credit, risk, and information constraints by giving participating farmers input loans for high-quality seeds and fertilizer, crop insurance, and training on improved farming practices.[12] Both programs were successful and led to increases in output, yields, and profits—but these kinds of programs are quite expensive to administer.
More Evaluation Needed
Despite limited research in the coffee sector, training, certification, and cash transfer programs show very mixed results. This is likely due to the simultaneous constraints farmers face, on both the supply and the demand side. An intervention that only provides training or information may not be sufficient to raise productivity and income in a significant way; more promising results seem to come from bundled interventions, i.e., programs that target multiple constraints to productivity at once. These seem to be effective in improving quality, yields, and profits both in the agricultural sector and in the coffee sector.
One question that remains is whether these bundles offer “good value for money.” Previous research has not been able to identify the effects of individual components of these programs. On one hand, standardized bundled programs can be expensive, but they may allow the program’s operator to implement its program in different locations without having to customize it for each new context. On the other hand, it could be expensive to tailor simpler programs to each new context. The cost-effectiveness of these bundled programs, therefore, depends on the balance between the cost of including too many components and the cost of adapting the program to each new context through market research.
Most of all, however, our research revealed the shortage of high-quality evidence in this area. Though hundreds—perhaps thousands—of certifications, development programs, buyer-driven programs, and other interventions exist to address livelihood concerns in coffee producing areas, very few studies on the effectiveness of these programs exist. If we hope to understand the effectiveness of the programs in which our industry is investing, we must increase the number of studies that critically evaluate the impacts of these programs.◇
DAVIDE DEL PRETE is an associate professor of economics at the University of Naples Parthenope and a research affiliate at the Food and Agriculture Organization of the United Nations. ROCCO MACCHIAVELLO is a professor of managerial economics and strategy at the London School of Economics and acts as the International Growth Centre’s Lead Academic (Myanmar). PETER GIULIANO is the Executive Director of the Coffee Science Foundation.
The authors would like to thank research assistant AISHWARYA REBELLY, whose excellent work greatly benefited this literature review project, and participants in the SCA’s Green Coffee Summit 2022 for their insightful comments.
The full literature review, commissioned by the Coffee Science Foundation with the support of a generous grant from Keurig Dr Pepper, is anticipated to publish in late 2023 and will be available at both the coffeescience.foundation and sca.coffee/news.
References
[1] Arthur Bluoin, Carl Cervone, David Del Prete, and Rocco Macchiavello, “Resource Misallocation and Gender Norms: Evidence from the Coffee Sector,” Mimeo (2022). Based on Enveritas data.
[2] Tasso Adamopoulos and Diego Restuccia, “Geography and Agricultural Productivity: Cross-Country Evidence from Micro Plot-Level Data,” The Review of Economic Studies 89, no. 4 (2022): 1629–1653.
[3] Alain de Janvry and Elisabeth Sadoulet, “Using Agriculture forDevelopment: Supply-and Demand-Side Approaches,” World Development 133 (2020): 105003.
[4] Uwe Deichmann, Aparajita Goyal, and Deepak Mishra, “Will Digital Technologies Transform Agriculture in Developing Countries?,” Agricultural Economics 47, S1 (2016): 21–33.
[5] Gashaw Tadesse Abate, Tanguy Bernard, Mekdim D. Regassa, and Bart Minten, “Improving Coffee Productivity in Ethiopia: The Impact of a Coffee Tree Rejuvenation Training Program on Stumping,” International Food Policy Research Institute (2021) / Esther Duflo, Daniel Keniston, Tavneet Suri, and Céline Zipfel, “Chat Over Coffee? Technology Diffusion Through Social and Geographic Networks in Rwanda,” Mimeo (2020).
[6] Overseas Development Institute, “Cash Transfers: What Does the Evidence Say?” (2016), accessed at https://cdn.odi.org/media/documents/11316.pdf.
[7] GiveDirectly, “Cash Crop: Evaluating Large Cash Transfers to Coffee Farming Communities in Uganda,” (2019), https://www.givedirectly.org/wp-content/uploads/2019/06/Cash_Crop_Ugandan_CoffeeRCT.pdf.
[8] Raluca Dragusanu, Eduardo Montero, and Nathan Nunn, “The Effects of Fair Trade Certification: Evidence from Coffee Producers in Costa Rica,” Journal of the European Economic Association 20, no. 4 (2022): 1743–1790.
[9] Rocco Macchiavello and Josepa Miquel-Florensa, “Buyer-Driven Upgrading in GVCS: The Sustainable Quality Program in Colombia,” Mimeo (2019).
[10] Lori A. Beaman, Dean Karlan, Bram Thuysbaert, and Christopher Udry, “Self‐Selection into Credit Markets: Evidence from Agriculture in Mali,” Yale Economics Department Working Paper No. 135 (2015); Michael R. Carter, Rachid Laajaj, and Dean Yang, “The Impact of Voucher Coupons on the Uptake of Fertilizer and Improved Seeds: Evidence from a Randomized Trial in Mozambique,” American Journal of Agricultural Economics 95 (2013): 1345–1351.
[11] Beaman et al. (2015).
[12] One Acre Fund.
We hope you are as excited as we are about the release of 25, Issue 20. This issue of 25 is made possible with the contributions of specialty coffee businesses who support the activities of the Specialty Coffee Association through its underwriting and sponsorship programs. Learn more about our underwriters here.