ajae_1294.tex Field Experiments in Food and Resource Economics Research (Brian Roe, Ohio State University, Organizer) CAN FIELD EXPERIMENTS RETURN AGRICULTURAL ECONOMICS TO THE GLORY DAYS? DAVID H. HERBERICH, STEVEN D. LEVITT, AND JOHN A. LIST As we contemplate the state of academic re- search in the agricultural economics commu- nity, there are two facts that are difficult to overlook. First, over roughly the first half of the twentieth century, scholars working on agri- cultural issues were the toast of the academy, setting forth an empirical research agenda that served to influence social and natural scientists profoundly. The seminal contributions made in these early years continue to shape important academic and policy debates. A second fact re- mains that this dominance has waned in the past several decades, with major empirical ad- vances now more likely in sister fields, such as labor economics or industrial organization. One piece of ocular evidence showing this decline is shown in figure 1, which reports the share of papers devoted to agricultural top- ics in four top economic journals over the period 1911–2000, quinquennially.1 In the ear- liest years more than 10% of papers in these four top journals were devoted to agriculture, with that share steadily falling through World War II. The share stayed roughly constant for David H. Herberich is a graduate student and research professional at University of Maryland and University of Chicago, Becker Cen- ter. Steven D. Levitt is the William B. Ogden Distinguished Service Professor, and John A. List is a professor, Department of Eco- nomics, University of Chicago, and NBER. Discussions with Bruce Gardner, Richard Just, and Marc Nerlove considerably enhanced the content of this paper. As usual, the authors are responsible for any errors, however. This article was presented in an invited paper session at the ASSA annual meeting in San Francisco, CA, January 2009. The articles in these sessions are not subjected to the journal’s standard refereeing process. 1 The journals are the American Economic Review, Economet- rica, Journal of Political Economy, and Quarterly Journal of Eco- nomics. For a more detailed description of our methodology for constructing figure 1, see the online appendix Herberich, Levitt, and List (2009). We understand that many approaches could be used to measure success of a field, and indeed, even under our approach one might argue that we include the wrong journals (Economic Journal, for example, played a significant role in the twentieth century and remains important today). We suspect that most, if not all, reasonable strategies will yield the general spatial pattern observed in figure 1. the next twenty years, before declining again to less than 1% of all papers by the end of the sample. In this study, we hypothesize on why agri- cultural economics played such a prominent role in economics for a long time, and why its importance has diminished more recently. We then provide one solution as to how agri- cultural economists can reverse this troubling trend and potentially return to a central place on the frontier of economic science: more ex- tensive use of field experiments. We view our solution as stemming from the roots of farm- ing, harkening back to Ralph Waldo Emerson who wrote: “The glory of the farmer is that, in the division of labors, it is his part to create.” Empirical Contributions of the Twentieth Century In this section, we briefly detail a select set of contributions that we view as seminal within the area of agriculture. From the outset, we concede that we cannot do this field justice due to space constraints so we limit our focus to briefly summarizing contributions of agri- cultural economists over the twentieth cen- tury. A useful starting point is the work of Fisher and Neyman, who were the first to conceptualize randomization as a key ele- ment of the experimental method in the 1920s and 1930s in answering important economic questions regarding agricultural productiv- ity (e.g., Splawa-Neyman 1990[1923]; Fisher 1926). Since Levitt and List (2008) discuss these contributions, we focus our study on sub- sequent decades, which witnessed agricultural economists making fundamental advances in regression analysis, including problems in- volving errors-in-variables and identification Amer. J. Agr. Econ. 91 (Number 5, 2009): 1259–1265 Copyright 2009 Agricultural and Applied Economics Association DOI: 10.1111/j.1467-8276.2009.01294.x 1260 Number 5, 2009 Amer. J. Agr. Econ. Note: The four Journals used are The Quarterly Journal of Economics, The American Economic Review, The Journal of Political Economy and Econometrica (begins 1933).Article by article search was done to determine if articles were related to Agriculture. Rejoinders, replies, restatements, comments orcorrections are omitted. The data was aggregated to 5-year periods begining in 1911 and running through 2000. Figure 1. Share of agricultural articles in four top economic journals problems concerning estimating elasticities of supply and demand (Fox 1986). The avail- ability of agricultural data during these years and the relevance of the estimation issues to agricultural questions are two possible expla- nations for why agricultural economists were early in addressing these issues. Other Early Empirical Contributions of the Twentieth Century Fox (1986) provides an important account of contributions that agricultural economists made to the areas of statistics and econo- metrics in the early half of the twentieth century. He mentions eight main agricultural economists during this time period: Louis H. Bean, Mordecai Ezekiel, Bradford B. Smith, Howard R. Tolley, Henry A. Wallace, Fred- erick V. Waugh, Holbrook Working, and E.J. Working as well as one economist working with agricultural commodities, Henry Schultz. A majority of these economists contributed to the field by playing a role in correctly address- ing identification issues resulting from sup- ply and demand analysis of agricultural data. However, more general work within multi- ple regression estimation was also being done by these scholars. For example, Tolley and Ezekiel (1923) initiated work toward expand- ing least squares estimation to multiple regres- sion. Ezekiel (1930) and Bean (1929) utilized graphical analysis to consider nonlinear im- pacts. Waugh (1935) increased the applicability of multiple regression by deriving matrix alge- bra computations. Working (1925) and Schultz (1925) recognized the attenuation bias that re- sults from the measurement error as early as 1925. Agricultural economists remained among the most prominent in the profession during the 1950s and 1960s. A pair of University of Chicago economists, Theodore Schultz and D. Gale Johnson, made lasting contributions to the field during this time period. Schultz’s early work dealt with agriculture in developed coun- tries, including issues of farm management. The work for which he was awarded the Nobel Prize in 1979, however, dealt with agriculture in developing countries. In particular, his re- search challenged the conventional view that traditional agriculture was not amenable to economic analysis because it was driven by tra- dition and culture rather than rational choices (Schultz 1964).2 Johnson’s contributions to agricultural economics spanned decades, be- ginning with his landmark analysis of the economics of share-cropping contracts (John- son 1950). His influential 1973 book World Agriculture in Disarray argued that govern- ment product and trademark interventions on 2 On arriving at the University of Chicago, one of the authors of this paper (Levitt) inherited the office that Ted Schultz had oc- cupied for decades. Upon hearing that Levitt was using Schultz’s office, the economist Sherwin Rosen lamented, “It’s too bad they gave you Ted’s office—all the good ideas in that office are already used up.” For a more detailed discussion of Schultz’s life and eco- nomic contributions, see Johnson (1999). Herberich, Levitt, and List Field Experiments in Agricultural and Resource Economics 1261 behalf of farmers had provided little real ben- efit to farmers. Even as an octagenarian, John- son continued to publish in leading economics journals (e.g., Johnson 2000). Other major players in this era include Marc Nerlove and Zvi Griliches. The collection of Nerlove’s papers in economics covers an extraordinarily wide range of topics, from adaptive expectations to spectral analysis to efficient methods of pooling time series of cross sections, but much of his work had agri- cultural applications or considerable implica- tions for agricultural analysis (e.g., Nerlove 1958; Balestra and Nerlove 1966). Although Zvi Griliches, who won the Clark medal just four years before Nerlove, is not principally thought of as an agricultural economist today, virtually all of his early work was on agri- cultural topics. This work included analyses of the demand for fertilizer and other inputs and most notably the diffusion of hybrid corn (Griliches 1957), which is the single most cited of his more than 200 papers. On the production economics front, Marschak and Andrews (1944) was seminal in combining first-order conditions with the production problem for estimation. Mundlak made important contributions along this line considering whether such systems involved simultaneous equation bias (Mundlak and Hoch 1965). This work provided a starting point for incredibly insightful refinements, particularly relevant to later agriculture work (e.g., Just and Zilberman 1983; Chambers and Just 1989). Another related area of study that has been influential more carefully considers implications of risk, such as Just and Pope (1978) and Chambers and Quiggin (2002). As mentioned, the availability of agricul- tural data as well as the relevance of the issues mentioned above to the types of ques- tions approached in agricultural economics are two possible explanations for why agricultural economists were early in addressing these is- sues. Further, during these formidable years, one aspect that separated the work of agri- cultural economists from the work of their economic counterparts may have been the former’s willingness to combine a strong theo- retical component with empirical analysis. As pointed out by Fox, Keynes lamented in 1917 that “to some economists the very idea of a mathematical treatment of economic problems is not only repugnant, but seems even absurd;” while Leontief assessed agricultural economics as “an exceptional example of a healthy bal- ance between theoretical and empirical analy- sis” (Leontief 1971, p. 5). Regardless of the underlying explanation, due to the cutting edge nature of the contributions in this field during this time period, major journals were common publishing outlets for agricultural work as noted earlier in figure 1. Later Empirical Contributions In aggregate, figure 1 suggests that agricul- tural economists of more recent generations have struggled to achieve the publishing suc- cess of their forebearers. One partial explana- tion, among many others, for this decline is that there has been an increased emphasis in top journals on empirical work that features plausibly exogenous sources of identification (i.e., “natural experiments”); this research ap- proach has been less frequently used by agri- cultural economists. There are, of course, many notable exceptions. For instance, Knoeber and Thurman (1994) use a plausibly exogenous shift in the contractual form of the incentive scheme facing producers of broiler chickens. A firm that purchased broilers switched from rewards based on the ordinal rank across pro- ducers to one that rewarded producers based on performance relative to the mean. Using this shift in compensation structures, the au- thors test the predictions of tournament the- ory, finding the data consistent with all of the testable predictions. A Path Forward Clearly, over the last third of the twenti- eth century, agricultural economists have not achieved the same level of publication success in these top journals as they had in earlier years. While our empirical analysis is limited to top general interest journal publications, we conjecture that this decline would be true un- der any of a broad number of metrics—faculty positions, quality and quantity of PhD stu- dents, share of federal grant monies, share of overall citations, etc. Our working hypothesis is that the combination of richer data sets com- ing on-line in sister fields—such as the large- scale data sets used today in labor economics— and issues in agricultural economics losing their relative cachet set forth the wheels of mo- tion to send senior scholars packing for more fertile fields and limited the attractiveness of the field of agricultural economics to the best young minds. There are several ways to proceed as a field. In one case, we can simply allow the market to 1262 Number 5, 2009 Amer. J. Agr. Econ. run its course, and perhaps in time our area will again become en vogue, perhaps in part driven by policy relevance, media relevance, or richer data sets of some sort being introduced. In some sense, we are observing this phenomenon now with the recent surge of research in global climate change amongst economists. A more active approach is to consider past successes and build on what seemed to work decades ago. As Levitt and List (2008) note and is recalled above, what might well be consid- ered the dawn of “field” experimentation was the work of Neyman and Fisher. Unfortu- nately, researchers in farm management in the 1950s and 1960s never connected their knowl- edge of the experimental method with labora- tory experiments using human subjects, which was at the very same time being pioneered by Vernon Smith at Purdue University (e.g., Smith 1962; this might be one reason for the subsequent demise of the farm management movement). Yet, there now might be a new train at the de- pot, perhaps providing agricultural economists with a unique second chance. While laboratory experiments have dominated the experimen- tal landscape in economics, the past decade has witnessed a significant surge in studies that gather data via field experiments. Field experi- ments in economics occupy an important mid- dling ground between laboratory experiments and studies making use of naturally occurring field data (see List 2006; Levitt and List 2008). This is beneficial because, on the one hand, economic theory is inspired by behavior in the field, so we would like to know if results from the laboratory domain are transferable to field environments. Further, field experiments al- low a search for similar causal effects using different identification assumptions than the strict assumptions necessary to achieve identi- fication using naturally occurring data. Harrison and List (2004) propose six fac- tors that can be used to determine the field context of an experiment: the nature of the subject pool, the information held by subjects, the commodity, the task or trading rules ap- plied, the stakes, and the environment in which the subjects operate. Using these factors, they discuss a broad classification scheme for field experiments, shown in figure 2, which helps or- ganize one’s thoughts about the factors that might be important when moving from the lab to the field. To date, field experiments across each of the three field experimental classifications—artifactual field experiments, framed field experiments, and natural field experiments—have shed insights on diverse areas through tests that pit neoclassical the- ory against prospect theory, that explore issues in cost/benefit analysis and preference elicita- tion, that explore competitive market theory in the field of alternative incentive schemes in developing nations, as well as tests of auction theory, the theory of private provision of public goods and of information assimilation among professional financial traders.3 So how does this growth in field experiments lend itself to agricultural economists? We speculate that the future of field experimen- tal methods can be importantly engineered by agricultural and resource economists. As Levitt and List (2008) discuss, one lesson learned from this most recent surge of field experimental research is that the researcher carrying out field experiments faces a set of challenges different from those that arise either in conducting laboratory experiments or relying on naturally occurring variation. The field experimenter does not exert the same degree of control over real markets as the scientist does in the lab. Yet, unlike an empiricist who collects existing data, the field experimenter is in the data generating business rather than data collection. Consequently, con- ducting successful field experiments demands a different set of skills from the researcher: the ability to recognize opportunities for experi- mentation hidden amidst everyday phenom- ena, an understanding of experimental design, knowledge of economic theory to motivate re- search and interpret empirical results, and pos- session of interpersonal skills to manage what are often complex sets of relationships involv- ing parties to an experiment. In this manner, agricultural economists have a comparative advantage in several factors in- strumental in conducting cutting-edge field ex- perimental research. First, existing extension programs provide specialists with a unique view of important everyday phenomena such as proposals for auctions to preserve conserva- tion land, new land use policies, insurance mar- kets, markets to hedge risk, and demand for new products. Many times, such ideas present themselves on the doorstep of extension spe- cialists long before they reach the “Ivory Tower.” Second, extension managers have the requisite interpersonal skills and knowledge of existing markets and market players to man- age the complex set of relationships involved in 3 See List’s website www.fieldexperiments.com, which contains a catalog of field experiments in these and several related areas. Herberich, Levitt, and List Field Experiments in Agricultural and Resource Economics 1263 Controlled Data Naturally Occurring Data ________________________________________________________________________ Lab AFE FFE NFE NE, PSM, IV, STR Lab: Lab experiment AFE: Artefactual field experiment FFE: Framed field experiment NFE: Natural field experiment NE: Natural experiment PSM: Propensity score estimation IV: Instrumental variables estimation STR: Structural modeling Note: This figure takes the field experimental classification scheme of Harrison and List (2004) and places it within other more traditional empirical approaches. Figure 2. Field experiments carrying out a clean experimental test. Finally, such an approach might better link extension specialists with theorists and statisticians who are trained in complementary research areas, yielding a much greater output than the sum of any of the inputs could yield in isolation. As in all such related cases, there is certainly avail- able human capital that will flock to such joint endeavors once the value of field experimen- tation is better recognized in this area. Such a research agenda, besides fostering important research links, has the capacity to lend important empirical insights into the types of theoretical models that best fit the data. Equally as important will be those cases in which the models are refuted by the data—in such instances, because the field experiments are able to collect enough facts to help con- struct new economic theories, there will be a healthy give and take between empiricists and theorists. Additionally, the field experiments in this area can measure key parameters that pro- vide economists with useful information and might lend insights into policymaking discus- sions. The underpinnings of such an approach are now being formed, as recent work across all three field experimental classifications in fig- ure 2 has begun in a fruitful direction.4 For 4 We only discuss a few here, but we would be remiss not to mention the work of Binswanger (1981), and the more recent work of Jayson Lusk (see, e.g., Lusk, Norwood, and Pruitt 2006). example, Schechter (2007b) evaluates risk and trust behavior using artifactual field experi- ments, finding that risk aversion explains a sub- stantial amount of decisions made by her group of Paraguayan farmers. Schechter (2007a) uti- lizes surveys that were run in conjunction with the experiments in Schechter (2007b) to pro- vide an interesting discovery of theft deter- rence: gift-giving, resulting in the formulation of a dynamic limited-commitment model on theft. Framed field experiments with farmers have also been used to inform policymakers, such as the auction work done by Cummings, Holt, and Laury (2003). The aforementioned au- thors began their research in response to The Flint River Drought Protection Act passed in Georgia in 2000, which mandated that an auction-like process for foregoing irrigation be implemented in years of drought. By running experimental auctions with farmers prior to the actual irrigation auctions, they were able to make suggestions on the auction design to make it more competitive and at the same time better understood by the participants. Examples of natural field experiments in this area also exist. For example in a partnership with the management of a leading fruit farm in the United Kingdom, Bandiera, Barankay, and Rasul (2005, 2006) use a natural field exper- iment to explore interesting economic ques- tions. Their subjects are farm workers, whose main task is to pick fruit. In one experiment, 1264 Number 5, 2009 Amer. J. Agr. Econ. workers were paid according to a relative in- centive scheme that provided a rationale for cooperation as the welfare of the group was maximized when workers fully internalize the negative externality that their effort placed on others. Bandiera, Barankay, and Rasul (2005) find that behavior is consistent with a model of social preferences when workers can be moni- tored, but when workers cannot be monitored, pro-social behaviors disappear. In their 2006 study, they find that individuals learn to coop- erate over time, both from their experience and from the experience of others. Together, these advances help us to understand workplace incentives. Conclusion Agricultural economics stands today at a cross- roads. Scholars working on agricultural issues once held a prominent role in conceptualiz- ing key empirical approaches that impacted the economics profession more generally. Yet the past several decades have witnessed other fields taking over central roles in the profes- sion. We hypothesize that this relative decline is due to the combination of richer data sets coming on-line in sister fields and the issues of agricultural economics losing their appeal. Yet, we do not view the situation as dire; rather, we advocate an active approach to returning to the roots of agricultural economics to help regain our relative footing. In this manner, we view agricultural economists possessing an enviable position to reestablish their place at the frontier of economic science. Though several years have passed since the Farm Management Move- ment was crushed, the time is ripe for theo- rists to combine with extension specialists to lend insights into both normative and posi- tive issues of the day. Years ago, there was not the methodological coherence and direc- tion that could be used by experimentalists on the farm; those days have passed. Agricul- tural economists can more easily combine the- ory and empirical work, taking advantage of their unique opportunities to execute field ex- periments. Whether such experiments revolve around exploring optimal incentive schemes of farm laborers in Europe, measuring de- mand elasticities for Bt cotton in the South, or examining farmers’ risk preferences in the Midwest, there is much to be done. In this way, agricultural economists have a clear route back to share the status of their subjects: substantial creation. References Balestra, P., and M. Nerlove. 1966. “Pooling Cross Section and Time Series Data in the Estimation of a Dynamic Model: The Demand for Natural Gas.” Econometrica 34(3):585–612. Bandiera, O., I. Barankay, and I. Rasul. 2005. “Social Preferences and the Response to In- centives: Evidence from Personnel Data.” Quarterly Journal of Economics 120(3):917– 62. ——. 2006. “The Evolution of Cooperative Norms: Evidence from a Natural Field Experiment.” B.E. Journal of Economic Analysis & Policy 6(2):Article 4. Bean, L.H. 1929. “A Simplified Method of Graphic Curvilinear Correlation.” Journal of the Amer- ican Statistical Association 24(168):386–97. Binswanger, H. 1981. “Attitudes toward Risk: Theo- retical Implications of an Experiment in Rural India.” Economic Journal 91:867–91. Chambers, R.G., and R.E. Just. 1989. “Estimating Multi-output Technologies.” American Journal of Agricultural Economics 7(4):980–95. Chambers, R.G., and J. Quiggin. 2002. “Optimal Producer Behavior in the Presence of Area- Yield Crop Insurance.” American Journal of Agricultural Economics 84(2):320–34. Cummings, R.G., C.A. Holt, and S.K. Laury. 2003. “Using Laboratory Experiments for Policy Making: An Example from the Georgia Irriga- tion Reduction Auction.” Working paper No. 06-14, Andrew Young School of Policy Studies Research Paper Series, September. Ezekiel, M. 1930. “The Sampling Variability of Linear Curvilinear Regressions: A First Ap- proximation to the Reliability of the Results Secured by the Graphic ‘Successive Approx- imation’ Method.” Annals of Mathematical Statistics 1(4):275–315. Fisher, R.A. 1926. “The Arrangement of Field Tri- als.” Journal of the Ministry of Agriculture of Great Britain 33:503–13. Fox, K. 1986. “Agricultural Economists as World Leaders in Applied Econometrics, 1917–1933.” American Journal of Agricultural Economics 68(2):381–6. Griliches, Z. 1957. “Hybrid Corn: An Exploration in the Economics of Technological Change.” Econometrica 2(4):501–22. Harrison, G.W., and J.A. List. 2004. “Field Ex- periments.” Journal of Economic Literature 42(4):1009–55. Herberich, D.H., S.D. Levitt, and J.A. List. 2009. “AJAE Appendix: Can Field Experiments Re- turn Agricultural Economics to the Glory Days?” Unpublished Manuscript. Available at: http://ageconsearch.umn.edu. Herberich, Levitt, and List Field Experiments in Agricultural and Resource Economics 1265 Johnson, D.G. 1950. “Resource Allocation under Share Contracts.” Journal of Political Economy 58(2):111–23. ——. 1973. World Agriculture in Disarray. London: Macmillan. ——. 1999. “Theodore William Schultz.” Biograph- ical Memoirs. 77:303–13. ——. 2000. “Population, Food, and Knowledge.” American Economic Review 90(1):1–14. Just, R.E., and R.D. Pope. 1978. “Stochastic Spec- ification of Production Functions and Eco- nomic Implications.” Journal of Econometrics 7(1):67–86. Just, R.E., and D. Zilberman. 1983. “Stochastic Structure, Farm Size and Technology Adoption in Developing Agriculture.” Oxford Economic Papers 35(2):307–28. Keynes, J.N. 1917. The Scope and Method of Political Economy. London: Macmillan. Knoeber, C.R., and W.N. Thurman. 1994. “Test- ing the Theory of Tournaments: An Empirical Analysis of Broiler Production.” Journal of La- bor Economics 12(2):155–79. Leontief, W. 1971. “Theoretical Assumptions and Nonobserved Facts.” The American Economic Review 61(1):1–7. Levitt, S., and J. List. 2008. “Field Experiments in Economics: The Past, The Present, and The Future.” NBER Working paper 14356, NBER, Cambridge, MA. List, J.A. 2006. “Field Experiments: A Bridge be- tween Lab and Naturally Occurring Data.” Advances in Economic Analysis and Policy 6(2):1747. Lusk, J.L., B. Norwood, and R. Pruitt. 2006. “Con- sumer Demand for a Ban on Subtherapeutic Antibiotic Use in Pork Production.” American Journal of Agricultural Economics 88:1015–33. Marschak, J., and W.H. Andrews. 1944. “Random Simultaneous Equations and the Theory of Production.” Econometrica 12(3,4):143–205. Mundlak, Y., and I. Hoch. 1965. “Consequences of Alternative Specifications in Estimation of Cobb-Douglas Production Functions.” Econo- metrica 33(4):814–28. Nerlove, M. 1958. “Adaptive Expectations and Cob- web Phenomena” Quarterly Journal of Eco- nomics 72(2):227–40. Schechter, L. 2007a. “Theft, Gift-giving, and Trust- worthiness: Honesty Is its Own Reward in Rural Paraguay” American Economic Review 97(5):1560–82. ——. 2007b. “Traditional Trust Measurement and the Risk Confound: An Experiment in Rural Paraguay.” Journal of Economic Behavior & Organization 62:272–92. Schultz, H. 1925. “The Statistical Law of Demand as Illustrated by the Demand for Sugar.” Journal of Political Economy 33(5):481–504. ——. 1928. The Statistical Laws of Demand and Sup- ply, with Special Application to Sugar. Chicago: University of Chicago Press. Schultz, T. 1964. Transforming Traditional Agricul- ture. New Haven: Yale University Press. Smith, V. 1962 “An Experimental Study of Com- petitive Market Behavior.” Journal of Political Economy 70(2):111–37. Splawa-Neyman, J., 1990[1923]. “On the Appli- cation of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9.” Statistical Science 5(4):465–472. Translated and edited bv D.M. Dabrowska & T.P. Speed from the Polish original, which appeared in Roczniki Nauk Rolniczyc, Tom X (1923): 1–51 (Annals ofAgricultura1 Sciences). Tolley, H.R., and M.J.B. Ezekiel. 1923. “A Method of Handling Multiple Correlation Problems.” Journal of the American Statistical Association 18(144):993–1003. Waugh, F.V. 1935. “A Simplified Method of Deter- mining Multiple Regression Constants.” Jour- nal of the American Statistical Association 30(192):694–700. Working, H. 1925. “The Statistical Determination of Demand Curves.” The Quarterly Journal of Economics 39(4):503–43.