Companies spend millions (perhaps billions?) of dollars every year surveying consumers to figure out want they want. Environmental, health, and food economists do the same to try to figure out the costs and benefits of various policies. What are people willing to pay for organic or non-GMO foods or for country of origin labels on meat? These are the sorts of questions I'm routinely asked.
Here's the problem: there is ample evidence (from economics and marketing among other disciplines) that people don't always do what they say they will do on a survey. A fairly typical result from the economics literature is that the amount people say they are willing to pay for a new good or service is about twice what they'll actually pay when money is on the line. It's what we economists call hypothetical bias.
We don't yet have a solid theory that explains this phenomenon in every situation, and it likely results from a variety of factors like: social desirability bias (we give answers we think the surveyor wants to hear), warm glow, yea-saying, and self presentation bias (it feels good to support "good" causes and say "yes", and why not say we're willing to do something, particularly when there is no cost to doing so and it can make us look and feel good about ourselves), idealized responses (we imagine whether we'd ever buy the good when we eventually have the money and the time is right, rather than answering whether we'd buy it here and now), strategy (if we think our answers to a survey question can influence the eventual price that is charged or whether the good is actually offered, we might over- or under-state our willingness to buy), uncertainty (research suggest a lot of the hypothetical bias comes from those who say they aren't sure about whether they'd buy the good), among other possible reasons.
What to do?
Various fixes have been proposed over the years.
- Calibration. Take responses from a survey and reduce them by some factor so that they more closely approximate what consumers will actually do. The problem: calibration factors are unknown and vary across people and goods.
- Cheap talk. On the survey, explain the problem of hypothetical bias and explicitly ask people to avoid it. The problem: it doesn't always "work" for all people (particularly experienced people familiar with the good), and there is always some uncertainty over whether you've simply introduced a new bias.
- Certainty scales. Ask people how sure they are about their answers, and for people who indicate a high level of uncertainty, re-code their "yes" answers to "no". The problem: the approach is ad-hoc, and it is hard to know a priori what the cut-off on the certainty scale should be. Moreover, it only works for simple yes/no questions.
- Use particular question formats. Early practitioners of contingent valuation (an approach for asking willingness-to-pay popular in environmental economics) swear by a "double-bounded dichotomous choice, referendum question" which they believe has good incentives for truth telling if respondents believe their answers might actually influence whether the good is provided (i.e., if the answer is consequential). I'm skeptical. I'm more open to the use of so-called "choice experiments", where people make multiple choices between goods that have different attributes, and where we're only interested in "marginal" trade offs (i.e., whether you want good A vs. good B). There is likely more bias in the "total" (i.e., whether you want good A or nothing).
There is another important alternative. If the problem is that surveys don't prompt people to act as they would in a market, well, whey don't we just create a real market? A market where people have to give up real money for real goods - where we make people put their money where their mouth is? It is an approach I wrote about in the book Experimental Auctions with Jason Shogren and it is the approach I teach with Rudy Nayga, Andreas Drichoutis, and Maurizio Canavari in the summer school we have planned for this summer in Crete (sign up now!) It is an approach with a long history , stemming mainly from the work of experimental economists.
One of the drawbacks with the experimental market approach is that it is often limited to a particular geographic region. You've got to recruit people and get them in a room (or as people like John List and others have done, go to a real-world market already in existence and bend it to your research purposes).
Well, there's now a new option with particularly wider reach. Several months ago I was contacted by Anouar El Haji who is at the Business School at the University of Amsterdam. He's created a simple online platform he calls Veylinx where researchers can conduct real auctions designed to give participants an incentive to truthfully reveal their maximum willingness-to-pay. The advantage is that one can reach a large number of people across the US (potentially across the world). It's a bit like ebay, but with a much simpler environment (which researchers can control) with a clearer incentive to get people to bid their maximum willingness-to-pay.
One of the coolest parts is that you can even sign up to participate in the auctions. I've done so, and encourage you to do the same. Hopefully, we'll eventually get some auctions up and running that relate specifically to food and agriculture.