“Response rates on [USDA-National Agricultural Statistics Survey] crop acreage and production surveys have been falling in recent decades (Ridolfo, Boone, and Dickey, 2013). From response rates of 80-85 percent in the early 1990s, rates have fallen below 60 percent in some cases (Figure 1). Of even greater concern, there appears to an acceleration in the decline in the last 5 years or so, suggesting the possibility that this decline reflects a long-term permanent change.”
That's from an interesting (yet worrying) article by the USDA chief economist Robert Johansson along with Anne Effland, and Keith Coble at farmdocdaily.
Why does this matter?
Responses to these surveys form the basis of what we think we know about, for example, how much farmland is in production, how much corn vs. soybeans is planted in a given year, the extent to which wheat yields are trending upward, and more. It's hard to understate how much of what we think we know about the state of U.S. agriculture stems from these surveys. For examples, I used these data in my article in the New York Times to describe the gains in farm productivity over time; economists use the data to try to predict the possible effects of climate change on crop yields and farm profitability; the data are used to try to figure out how farmer's planting decisions respond changes in crop prices (which provide estimates of the elasticity of supply, which feed into various models that inform policy makers), and much more.
The concern with falling response rates is that the farmers who respond may be different than the one's who don't in a way that biases our understanding of crop acreage and production. The authors write:
“Reduced response rates can potentially introduce bias or error to the estimates released by USDA. For example bias may occur if higher yielding farms drop out. Reduced response will almost assuredly introduce error to the estimates making them noisier and randomly more inaccurate. This will be most noticeable in county estimates. ”
The authors go on to note that some farm program payments depend on county-level yield estimates (which the above note notes are now less reliable). As such, this isn't just some academic curiosity, but an issue that could literally affect millions of taxpayer dollars.
The problem of declining response rates isn't just with farmers. This paper, appropriately titled "Household Surveys in Crisis", points out it is an issue with other government surveys of households as well. These are the surveys that attempt to provide statistics on people's incomes, employment, and so forth.
The solutions to these problems are not obvious or easy. Here is the authors' take:
“Some research suggests that tailoring survey approaches to differing audiences within the survey population could improve response rates (Anseel et al., 2010). Other data sources like remote sensing, weather data, modeling, machine data, or integrated datasets may also be useful in providing additional information. NASS already makes use of some of these other data sources and methods in developing estimates, but as a supplement, not a replacement, for survey data. Further use of such sources is costly. For now, the best approach remains encouraging greater producer response.”