On the estimation of on-farm benefits of agricultural research
David J. Pannell
Agricultural and Resource Economics, University of Western Australia, Nedlands 6907
Abstract
The on-farm benefits of agricultural research or extension often depend in complex ways on the way that the new technology or information affects the farming system. The existence of system-wide impacts of research and extension is highlighted as a critical but neglected issue in economic evaluations of agricultural research. The issue is neglected in the available texts on agricultural research evaluation and in most applied evaluations. Given the difficulty and complexity of accurate benefit estimation, we see a renewed role for farm-level economic models (such as whole-farm linear programming models) in this area. The benefits of undertaking a more sophisticated and detailed analysis to estimate research benefits include not just greater accuracy but also greater credibility with researchers and greater relevance through representing factors which they perceive to be important. The paper discusses how, if such respect is engendered, a formal research evaluation can yield additional benefits by improving the design of research.
1. Introduction
Changing government philosophies and reduced public funding for agricultural research institutions (which reflect the changed philosophies) have combined to produce an unprecedented interest in formal economic evaluations of agricultural research. In Australia this is apparent in state Departments of Agriculture, the CSIRO and the national agricultural research funding bodies (the Rural Industry Research Corporations or RIRCs).
In applying Benefit Cost Analysis (BCA) to research, a range of types of information is needed. For farm-level research, the estimation of on-farm benefits is notable for being both critical to the analysis and difficult to accurately obtain. This very complex issue has often been handled in an oversimplified way. The first aim in this paper is to outline the risks of simplified approaches to estimation of farm-level benefits of research and to raise awareness of the utility of relatively detailed farm models in research evaluation.
The motivations of most research administrators in supporting economic evaluations of research are (a) to obtain evidence which will support cases they make externally to maintain current levels of funding in the face of threatened cuts, and (b) to help prioritise areas of research to identify low-return areas for cuts and to high-return areas and new opportunities for increased funding. However, the benefits of undertaking formal research evaluations are not confined to these areas. The second aim here is to suggest how to involve and influence biological and physical scientists so that research evaluations are successful and productive and so that they are most likely to generate the spin-off benefits which can occur.
2. The Difficulty of Estimating On-Farm Benefits of Research
Information needed to evaluate an agricultural research project using BCA includes the following components:
(a) The predicted (ex ante) or estimated (ex post) biological, technical and/or management changes from implementing research outcomes.
(b) Any negative or positive side effects (internal or external to the farm) resulting from implementation of the research. This would include any environmental externalities and price impacts from changes in supply or demand.
(c) Costs to the farm firm of implementing findings from the research.
(d) Given (a), (b), and (c), the potential economic benefits per hectare or per farm (net of costs to the farm firm but not of research costs).
(e) The scale of potential benefits: the number of hectares or farms potentially affected.
(f) The proportion of the potential scale for which adoption occurs, and the timing of the adoption.
(g) The probabilities of different levels of success from the research.
(h) Direct costs of undertaking the research over time.
(i) The discount rate.
In practice, the outcomes of BCAs of research are most sensitive to items (d), (e) and (f). Of these, the scale of potential benefits can usually be estimated with tolerable accuracy, but substantial uncertainty often surrounds farm-level benefits and adoption. In both cases, economists often resort to heroic simplifying assumptions of dubious validity. Improvements in assumptions regarding adoption will probably depend on further research on the relationship between characteristics of different research outcomes and the patterns of their adoption. The rest of this section and the next focuses on the estimation of on-farm benefits, outlining a case for more detailed, comprehensive and sophisticated modelling to estimate them. This focus on benefits to farmers is not meant to imply that benefits to others in the community (such as consumers or tax payers) should not be considered. Where public money is invested, it is reasonable to expect that benefits and costs to the community as a whole should be evaluated. However, in doing this, accurately estimating benefits to farmers will usually be particularly important and particularly difficult.
It is recognised that in calculating component (d), obtaining components (a) and (b) may not be straightforward. Scientists’ estimations are necessarily subjective and subject to uncertainty. There is uncertainty too in estimating (d). This paper is concerned with reducing this uncertainty about (d), which is much greater than commonly realised but which can be substantially reduced by the use of detailed bioeconomic models. Uncertainty about (d) arises for a variety of reasons, including the following:
In principle, net-on-farm benefits from research are approximated by changes in the level of producer surplus resulting from shifts in supply and demand curves (Mishan, 1982; Gittinger, 1982). In practice, the complexities listed above mean that shifts in supply are often extremely difficult to quantify accurately. In applied BCAs, one approach sometimes used to estimate shifts in supply is to ask scientists conducting the research to estimate the reductions in marginal cost per unit of production. For the reasons listed above, especially the last three, this is an extremely unreliable practice. Economic impacts often do not follow simply and directly (e.g. in direct proportion to) the biological impacts, so that relatively detailed analyses are necessary to obtain reasonably accurate estimates of farm-level economic benefits of research. For example, the relationship between pasture production and profit is extremely complex (Pannell and Panetta, 1986). The task of estimating economic impacts of research is generally so complex, even at the farm level, that it is well beyond the scientists’ competence. Consequently we should not ask scientists to go beyond providing estimates of the biological impacts of their research. Agricultural economists should themselves infer the economic impacts and they need sophisticated tools to do so. In the following section, this argument is explored for the different types of agricultural research listed in Table 1.
Table 1. Types of on-farm impacts of agricultural research
1. Improved technology
1.1 New enterprise (e.g. new legume crop species).
1.2 Increased production (e.g. new crop variety).
1.3 Decreased production costs (e.g. more efficient technology for application of chemicals)
1.4 Increased product quality (e.g. reduced contamination)
1.5 Reduced risk (e.g. a more stable yielding crop variety)
2. Information
2.1 More rapid adoption and/or a higher level of adoption of beneficial existing technology (e.g. demonstration trials for alternative farming systems)
2.2 Better management decisions leading to higher profit (e.g. development of decision support systems)
2.3 Reduced risk (e.g. improved climate forecasts)
3. Categories of Agricultural Research
The major division in Table 1 is between research which results in a new physical product or technology and research which produces information on an existing technology or for improved management decisions. It is recognised that there is not a clear-cut dichotomy (e.g. a new technology must be accompanied by information at least on how to apply it).
New enterprise (item 1.1)
Item 1.1 in Table 1 is perhaps the most complex due to interactions with existing farm products through substitution and competition for resources and possibly through biological complementary or competitive effects. It is also unusual in that there is no producer surplus from this product without the research and consequently the benefit of the research equals the whole producer surplus from this product minus its fixed costs.
To illustrate the impacts of biological interactions between products, Figure 1 shows some results from MIDAS, a whole-farm linear programming (LP) model of crop-livestock farming in Western Australia (Kingwell and Pannell, 1987). The graph shows the farm-level supply curves for wheat (smoothed by simple OLS regression) with and without the inclusion of lupins, a legume crop grown in rotation with wheat. The results simulate the impact on wheat supply of the introduction of lupins to the farm.
Figure 1. Wheat supply curves (Source: MIDAS model version EWM 91-4, Pannell and Bathgate, 1991)
Lupins affect the supply curve for wheat in three ways:
(a) To some extent lupins are grown instead of wheat, and this tends to move the wheat supply curve to the left.
(b) Wheat grown in rotation with lupins gives a higher yield than wheat following wheat, tending to move the supply curve to the right.
(c) Because of disease risks, it is not feasible to grow lupins in a rotation which does not include a cereal, so on soil types where lupins yield well, there is a tendency to grow wheat even if the wheat is not profitable in itself. This means that at low wheat prices, the availability of lupins tends to move the wheat supply curve to the right.
The combination of these factors means that the two supply curves cross (with and without lupins). There are several points to note about this example.
(a) In estimating the impact of a new enterprise, account must be taken of the implications for other enterprises.
(b) If other affected products have inelastic demand curves, it may also be necessary to estimate their supply shifts as a result of a new product. The nature of these supply shifts for other products is by no means obvious, especially in rotational farming systems.
(c) In order to understand the position of the supply curve for the new product and the shifts in supply for existing products, it is necessary to undertake a fairly detailed analysis, such as running a whole-farm optimisation model. This is illustrated in Figure 2 which shows supply curves for lupins calculated with and without allowance being made for biological interactions between lupins and other enterprises. The interactions include: fixation of atmospheric nitrogen by lupins, improved soil structure following lupins, reduced cereal disease levels following lupins, use of lupin grain and lupin crop residues as sheep feeds, and increased efficiency of machinery use. Figure 2 shows that a failure to represent these biological interactions leads to major errors in the calculation of economic benefits from lupins; the producer surplus calculated without interactions is very much smaller than the value calculated when interactions are considered.
Figure 2. Lupin supply curves (Source: MIDAS model version EWM 91-4, Pannell and Bathgate, 1991)
(d) In cases where the new product has an inelastic demand curve, the equilibrium price of the new product will be affected by the product's on-farm interactions with other enterprises via the effect of these interactions on the supply curve of the new product.
(e) If a detailed whole-farm optimisation model is used and the demand curve is highly elastic (e.g. for an exported product) there is no need to resort to the use of producer surplus to approximate changes in profit, because the analysis will provide profit calculations directly. (These clearly need to be weighted and aggregated to give industry level benefits). If the demand curve is inelastic, the whole-farm model can be used to estimate supply curves for individual representative farms, to be aggregated appropriately to give the industry supply curve. This can then be combined with the market demand curve to calculate aggregate producer and consumer surplus.
Increased production (item 1.2) and Decreased production costs (item 1.3)
Items 1.2 and 1.3 from Table 1 are equivalent from an economic point of view in that they both result in a shift in the supply curve downwards/to the right (e.g. Duncan and Tisdell, 1971). Most discussions of economic research evaluation have focused to a large extent on these two categories, especially with regard to estimation of benefits (e.g. Lindner and Jarrett, 1978; Edwards and Freebairn, 1981; Voon and Edwards, 1991). Nevertheless many of the other categories in Table 1 account for large levels of research expenditure.
For example, a large proportion of research conducted by state Departments of Agriculture in rural areas of Australia is in testing, measuring, and demonstrating existing technology, with the aim of generating benefits in category 2.1. Even where research involves technology which is new to a region, there is often a high demonstration/information component of the research which may affect the timing of adoption more than it affects the ultimate size of the supply shift. The narrow focus on these categories by some has resulted in some notable misguided attempts to squeeze other types of research into an analysis of benefits based solely on generation of a supply shift.
Despite the attention paid by economists to categories 1.2 and 1.3 of research benefits, the difficulty of estimating their magnitudes remains considerable. Even such apparently simple issues as the appropriate nature of the shift (parallel, divergent, or convergent) have been debated at length (e.g. Lindner and Jarrett, 1978; Rose, 1980) but not resolved. In practice many economists resort to Rose's expedient of assuming a parallel supply shift. This simple formula for what-to-do-when-you-don't-know-what-else-to-do, has become much more popular than it deserves and is now often applied uncritically. Or perhaps it is just that economists often don't know what else to do. The result is extremely large error margins - much greater than could be achieved by more detailed assessments at the farm level. The attraction of a detailed whole-farm LP model is that it avoids having to make simplistic and probably inaccurate assumptions about the supply curve, including its elasticity and how it shifts following research. This is achieved by directly representing the resource constraints, differences in resource qualities, technical relationships, alternative enterprises and their bio-physical interactions.
If the effects of 1.2 and 1.3 are sufficiently great to substantially increase the resource allocated to an enterprise, there will be a need to account for the substitution for other enterprises which is discussed in 1.1. There may also be biological interactions; research which increased nitrogen fixation by a legume crop would be expected to benefit both the legume crop and subsequently grown cereals. Again, these opportunity costs and biological interactions should be reflected in calculations of the change in producer surplus for the product which was the subject of the research. Correctly estimating their impact is straightforward in a detailed whole-farm LP model, but it is very difficult without such a tool, other than by heroic assumption.
Increased product sale price (item 1.4)
Evaluation of research which increases product prices, such as through higher product quality, has been discussed, for example, by Voon and Edwards (1992) and Alston et al. (1995). Again, however, the possible resulting interactions between enterprises have been relatively neglected. For example, the prime vehicle for increasing the protein content of Australian wheat has been through rotation with legume crops and pastures, and such rotations have a multitude of impacts. Apart from the obvious impact of nitrogen fixation by the legume, there may be yield increases for other reasons such as breaking the cereal disease cycle or improving soil structure. Choice of rotation also affects pasture density, legume crop disease, stubble management, weed control method and tillage method (Pannell, 1995). Correctly estimating the economic implications of these impacts is a non-trivial task, but can be achieved well using the type of whole-farm model advocated here.
Reduced risk (item 1.5)
Another area with a high profile in recent agricultural policy discussions in Australia is risk. With the demise of price stabilisation schemes for wheat and wool and the change of approach to drought policy (e.g. Simmons, 1993), broad-acre farmers are exposed to greater risks than previously. Most of the new research responding to this change has been in category 2.3, information for improved risk management. However it is also possible that development of new enterprises could yield benefits from reduced risks. This could occur through (a) the new technology generating a more stable income stream for the firm due to greater yield and/or price stability, (b) a negative correlation between price and yield for the new enterprise or (c) lack of correlation between income from the new enterprise and that from existing enterprises, so that there is scope for reducing risk through diversification.
In principle, benefits to farmers from reduced risks would be reflected in a supply shift (Newbery and Stiglitz, 1981) leading to greater producer and/or consumer surplus. However the difficulty of predicting the size of the shift is even greater than that for categories 1.1-1.4. A relatively detailed and sophisticated analysis may be necessary to obtain an estimate of the benefits which is even of the right order of magnitude. The analysis should be based on a standard decision theory framework, as presented by Anderson et al. (1977). It would explicitly represent the probabilities of different on-farm outcomes from the new information or technology, risk averse attitudes of farmers, and, where the research produces information, the way that such information influences farmers’ risky decision making (e.g., Kingwell, 1994).
Enhanced adoption (item 2.1)
Marshall and Brennan (1993) emphasise the importance of and high investment in this category of research. Available evidence indicates that most farmers eventually adopt available technologies if they are truly superior (Lindner, 1987; Marsh et al. 1995), so the main impact in this category is likely to be more rapid adoption rather than higher final levels of adoption. The technology or management system which is adopted sooner could be from any of the other categories of Table 1 (a new product, a cheaper production system, a higher quality product), so the accuracy of the evaluation is dependent on the accuracy with which the benefits of the new innovation are estimated.
Better management decisions (item 2.2)
The outcome of better information is likely to include a better allocation of resources and more profitable application rates or timing of inputs. Australian agricultural economics has a tradition of strength in a Bayesian approach to the value of information (e.g. Anderson et al., 1977) but this strength has not usually been applied in the area of research evaluation.
Reduced risk (item 2.3)
Research in this category includes research aimed at predicting weather conditions, simulation modelling to predict yields and economic research on optimal management in the face of production risk or market risk. Such research has two sources of benefit to farmers: a higher expected value of returns and a lower cost of risk for risk averse farmers. Both would shift the supply curve for the relevant product and so would have a measurable impact on producer surplus, which in this case approximate certainty equivalent rather than profit..
As example of the type of analysis needed to do justice to this category, Kingwell et al. (1993) use a whole-farm LP model based on discrete stochastic programming to estimate the economic benefits of using climatic information to adjust farm management decisions. Estimating these benefits without such a detailed and sophisticated tool would be extremely prone to error. In particular, without such a tool one could not accurately predict shifts in supply from research which provided such climatic information, except by chance.
Clearly, it is a considerable challenge to economists to identify all the impacts of research, positive and negative, and include them in the evaluation. My aim in this paper is not to dismally condemn common practice, but to raise awareness among research evaluators of the reality that the impacts of most agricultural research are numerous and complex. If we are to do justice to the task and win the confidence of researchers, we must do better than guessing at supply shifts, and we certainly cannot expect scientists to guess at supply shifts for us. We must treat research as having impacts on a complex farming system, examining the system in some detail to obtain valid estimates of research impacts.
Finally, to clarify the ideas put here, consider the relationship between a structural modelling approach (whole-farm LP or simulation) and a supply-demand equilibrium displacement approach. Even in situations where demand is inelastic, the structural modelling approach is the best way to estimate the position and form of the supply curve and the nature and extent of any shift in it. It is substantially better in practice than making simplistic assumptions about supply elasticities, functional forms and supply shifts, as is common in the equilibrium displacement approach. However to adequately account for the inelasticity of demand, the supply curve(s) estimated using the structural model should be employed in a standard equilibrium displacement approach to estimate changes in producer and consumer surpluses. In cases where demand is sufficiently elastic, so that consumer surplus can reasonably be disregarded, the structural modelling approach is theoretically equivalent to the equilibrium displacement approach. The change in profit (or perhaps certainty equivalent) estimated using the structural model is the theoretically correct measure of producer benefits, and there is no need to repeat the exercise in an equilibrium displacement analysis which, if it does anything, is likely to distort the estimated benefit by imposing restrictions on the functional form of supply or the nature of the supply shift.
4. Interacting with Scientists
This section focuses on the relationship between economists and bio-physical scientists and the importance of this relationship in generating benefits from research evaluation activities. It is closely related to the last section because the modelling approach advocated there is very helpful in promoting a positive and constructive relationship between the disciplines (Pannell, 1996a).
There are various ways in which a formal economic evaluation of research may result in benefits: (a) by culling relatively unbeneficial research and redirecting funds to relatively beneficial research; (b) by changing the emphasis or design of a research project; (c) by changing the variables which are measured; (d) by facilitating information flows between biological and physical scientists, economists, administrators, extension specialists and farmers; and (e) by providing an appropriate focus and paradigm for research leaders, research managers, and scientists. Although managers of the research institution may be primarily interested in (a), the potential benefits from (b), (c), (d), and (e) are at least as great. However (b) to (e) are mainly in the domain of the individual scientist, rather than the research administrator, and so are perhaps less visible and less appreciated than they should be.
The issue is similar to one in farm management. Some of the most successful farmers employ very simplistic decision rules in choosing a portfolio of enterprises, but are successful because of their expertise in technical aspects of crop or livestock production such that they consistently attain high yields. Similarly with research, even if research evaluation did not influence which research projects are conducted (as chosen by research administrators or funders), it may increase the value of the research which is conducted by changing the research design or the variables measured (as chosen by the scientists) . In both cases, success is not so much a matter of which broad strategy you pursue, but of how well you implement the chosen strategy.
The fact that choices made in the implementation of a particular research project are made almost solely by the researchers themselves highlights the importance of there being a positive and constructive relationship between economists and bio-physical researchers. This is an issue which has recently been addressed by several authors (e.g. Mullen, 1996; Padberg, 1990; Pannell, 1996a; Young, 1995). One important issue is ensuring that the researcher has confidence in and respect for the economist. A substantial contribution to this end can be made if the analysis is seen to include the complexities and subtleties outlined above. A simpler analysis is cheaper and quicker and it may even be reasonably accurate for some research, but it certainly has an adverse effect on the credibility of the results in the eyes of scientists, often for good reason. Moreover by dealing with parameters of interest to the scientist, the analysis is more relevant.
Another set of issues related to confidence includes matters of interpersonal relationships, human foibles and sensitivities, rather than economics. It is hoped that the following observations and suggestions based on experience will be of value to others. They are mainly concerned with the problem of extension of research evaluation to scientists in cases where the scientist is less than enthusiastic about participating in the evaluation. There are many possible reasons for this reticence, including a degree of compulsion, defensiveness, inexperience, or a bad previous experience. Defensiveness may arise because of perceived loss of power to the analyst, and uncertainty as to whether or not the analysis will support the scientist's preferred topic.
Whatever the cause, the symptom of the scientist's concerns is criticism of the analysis. Where the criticisms are of specific assumptions, this is a good thing to be encouraged and responded to by revising the analysis. Often it can be shown that an unwanted or unexpected result is not sensitive to the assumptions of which the scientist is critical. The key to convincing them is to get them to participate jointly in sensitivity analysis, examining plausible ranges for parameters of interest or concern to them.
On the other hand, where the fundamental basis of the analysis is attacked, some defensive arguments are needed. Some examples follow.
(a) to emphasise the constructive role of the analysis in generating ideas and in highlighting and prioritising knowledge gaps, rather than in providing absolute numerical results;
(b) to stress that the role of the analysis is to support what will always be a subjective decision about research directions. Without the analysis the decision would still rest on subjective data, but the data would not be subject to the broader review and criticism which is one of the benefits of making assumptions explicit;
(c) to use the argument that this information is not to be compared with perfect information but with the information available in its absence; and
(d) to stress the importance of transparency of assumptions and that no results should be taken seriously by anyone unless key assumptions are explicit so that decision makers can assess their validity.
Overall, it is most important to emphasise a constructive role, not a policing role. This is primarily achieved by one-to-one contact. The economists should aim to get right inside the research: to understand the science, and be seen to understand it, to actually look at the research if possible, preferably in the presence of the scientist(s). Even if the prospect of viewing biological experiments is boring, it can yield considerable benefits for the exciting part - the economic analysis. The scientist can get especially excited and effusive in the presence of the field trials, or laboratory, and inadvertently reveal unexpected gems of information. You will also get a better feel for the quality of the information likely to be generated. In addition your presence at the trial will promote trust and respect on the part of the scientist.
Communication of the economic approach is equally important. When numerical analyses are conducted, results should be shown to the participating scientists first to provide them with an opportunity to adjust their assumptions, or raise additional considerations, or constraints which they have overlooked. However, they must understand that key assumptions will then be open to critical review from peers and industry.
For a fully successful collaboration with research leaders and researchers, there needs to be more than good cooperation; joint "ownership" of the analysis is needed. Rather than being a defender of particular projects, the research leader or researcher needs to be interested in finding out where the highest returns to research are. This is an approach which can be foreign and difficult for many. The analysis may show that their pet area is unlikely to yield high returns, and that resources should be reallocated. Such findings can be palatable only where there is a stake in the analysis. With joint ownership of the analysis and involvement in direction setting, the researcher need not lose power with the introduction of research evaluation (unless power is narrowly perceived as keeping on doing more of the same).
Except when dealing with exceptional individual scientists who are naturally interested in economics, support from senior management is critical. There is little chance of a successful ongoing role for economists in research evaluation in an environment where the senior scientists or administrators are antagonistic to the activity. Scientists find it hard enough to find the time and motivation to participate without also having to deal with the wrath of their superiors for doing so. However, this would now seem to be less of a problem, in that senior management in most research organisations is now very conscious of the need for research evaluation. Senior managers in some of these organisations now perceive that a cultural shift is required in many researchers. Research evaluation is perceived as a means of helping the achievement of this because it questions the status quo and brings in the idea of research as an investment. It can fit in well with the new program structure (which is sweeping research organisations) with its focus on program outcomes and the organisation's mission statement.
Finally, this discussion has been predicated on a belief that the major benefits of evaluating research will come from choices made by scientists rather than by research funding bodies. Shumway (1981) expressed his concern about the potential adverse impact of compulsory BCAs on the essential creative inspiration of scientists. He concluded that, "Evaluation techniques which ... demand additional effort from the scientists in documentation and accountability for the system’s sake are doomed to dismal failure," (Shumway, 1981, p. 171).
5. Concluding Comments
There are now opportunities for agricultural economists to participate in the research evaluation and prioritisation process to an extent which has not previously been possible. It is essential that the work done is perceived as being relevant and valuable by scientists and research managers. It is argued here that in particular this means going to some effort to estimate the farm-level benefits of agricultural research. Placing meaningful dollar values on predicted outcomes of biological research is much more difficult and prone to error than many seem to realise, including many directly involved in conducting BCAs. The task realistically requires tools such as whole-farm mathematical programming models and/or decision theory models. For this reason, it should not be left to bio-physical scientists to estimate economic impacts of research (such as shifts in supply). Rather their role in research evaluation should be limited to estimating biophysical impacts of research, with economic interpretations, both at the farm and aggregate levels, provided by economists.
6. References
Alston, J.M., Edwards, G.W. and Freebairn, J.W. (1988). Market effects of research. American Journal of Agricultural Research 70, 281-88.
Alston, J.M., Norton, G.W. and Pardey, P.G. (1995). Science Under Scarcity: Principles and Practice for Agricultural Research Evaluation and Priority Setting, Cornell University Press, Ithaca.
Anderson, J.R., Dillon, J.L. and Hardaker, J.B. (1977). Agricultural Decision Analysis. Iowa State University Press, Ames.
Duncan, R. and Tisdell, C. (1971). Research and technical progress: The returns to producers. Economic Record 47, 124-29.
Edwards, G.W. and Freebairn, J. W. (1981). Measuring a Country’s Gains from Research: Theory and Application to Rural Research in Australia, Australian Government Publishing Service, Canberra.
Gittinger, J.P. (1982). Economic Analysis of Agricultural Projects, Johns Hopkins University Press, Baltimore.
Kingwell, R.S. (1994). Risk attitude and dryland farm management. Agricultural Systems 45: 191-202.
Kingwell, R.S. and Pannell, D.J. (Eds) (1987). MIDAS, A Bioeconomic Model of a Dryland Farm System, Pudoc, Wageningen.
Lindner, R.K. (1987). Adoption and diffusion of technology: An overview. In: Technological Change in Postharvest Handling and Transportation of Grains in the Humid Tropics, B.R. Champ, E. Highley and J. Remenyi (eds.), ACIAR, Proc. No. 19, pp. 144-151.
Lindner, R.K. and Jarrett, F.G. (1978). Supply shifts and the size of research benefits. American Journal of Agricultural Research 60, 48-58.
Marsh, S., Pannell, D.J. and Lindner, R.K. (1995). Impact of extension on adoption of lupins in Western Australia. Paper presented at the 39th Annual Conference of the Australian Agricultural Economics Society, Perth, February 14-16 1995.
Marshall, G. and Brennan, J.B. (1993). On Benefit-Cost Evaluation of Research Projects, Paper presented at the 37th Annual Conference of the Australian Agricultural Economics Society, University of Sydney, 9-11 February, 1993.
Mishan, E.J. (1982). Cost-Benefit Analysis (3rd ed.), Allen and Unwin, London.
Mullen, J. (1996). Why economists and scientists find cooperation costly. Review of Marketing and Agricultural Economics 64: 216-224.
Newbery, D.M.G. and Stiglitz J.E. (1981). The Theory of Commodity Price Stabilization, Clarenden, Oxford.
Padberg, D.I. (1990). Enlightened interaction between agronomy and agricultural economics. Journal of Production Agriculture 3: 170-173.
Pannell, D.J. (1995). Economic aspects of legume management and legume research in dryland farming systems of southern Australia. Agricultural Systems 49: 217-236.
Pannell, D.J. (1996a). Lessons from a decade of whole-farm modelling in Western Australia. Review of Agricultural Economics 18:373-383.
Pannell, D.J. (1996b). Toward a balance between strategic-basic and applied agricultural research. Proceedings, Global Agricultural Science Policy for the Twenty First Century, Contributed Papers, 26-28 August 1996, Melbourne, Australia, pp. 457-484.
Pannell, D.J. and Bathgate, A. (1991). MIDAS, Model of an Integrated Dryland Agricultural System, Manual and Documentation for the Eastern Wheatbelt Model Version EWM91-4, Miscellaneous Publication 28/91, Department of Agriculture, Perth, Western Australian.
Pannell, D.J. and Panetta, F.D. (1986). Estimating the on-farm cost of skeleton weed (Chondrilla juncea) in Western Australia using a wholefarm programming model. Agriculture, Ecosystems and Environment 17: 213-227.
Rose, R.N. (1980). Supply shifts and the size of research benefits: Comment. American Journal of Agricultural Research 62: 834-37.
Shumway, C.R. (1981). Subjectivity in ex ante research evaluation. American Journal of Agricultural Economics 63: 169-173.
Simmons, P. (1993). Recent developments in Commonwealth drought policy. Review of Marketing and Agricultural Economics 61: 443-54.
Varian, H.R. (1990). Intermediate Microeconomics, A Modern Approach, 2nd ed. Norton: New York.
Voon, J.P. and Edwards, G.W. (1991). The calculation of research benefits with linear and nonlinear specifications of demand and supply functions. American Journal of Agricultural Research 73: 415-20.
Voon, J.P. and Edwards, G.W. (1992). Research payoff from quality improvement: the case of protein in Australian wheat. American Journal of Agricultural Research 74: 565-72.
Young, D.L. (1995). Agricultural economics and multidisciplinary research. Review of Agricultural Economics 17: 119-129.
Citation: Pannell, D.J. (1999). On the estimation of on-farm benefits of agricultural research, Agricultural Systems 61(2): 123-134.
