Blog

How Reliable are Surveys of Public Opinion and Preference?

A day after the presidential election, it is useful to evaluate the usefulness of the surveys and polls that were used to predict election outcomes.  By and large, my assessment is that most of the polls got it about right.  Republicans complained a lot about state-level polls in Ohio and elsewhere saying that Democrats were being weighted too heavily, but in the end, it appears the pollsters had it right.  

Now turn to economics, where the latest issue of the Journal of Economic Perspectives just arrived in my inbox.  In the issue are three articles on the use of "contingent valuation", which is a survey method used to ask people how much they are willing to pay for some good - normally a public good (like the environment) that is not traded in a market setting.  The first two articles by Kling, Phaneuf, and Zhao and by Carson (here and here) are generally pro-contingent valuation, the last by Jerry Hausman at MIT is against.   

I've spent a lot of time studying contingent valuation and other consumer research methods.  The methods are far from perfect and they suffer from many well-known biases.  At the end of the day, however, my question is: what is the alternative?  

Here is Hausman's answer (footnote and references removed):

I am often asked what should be done given my view that contingent valuation should not be used. Should nonuse value be ignored? My view is that expert government agencies and Congress should make informed decisions and enact regulations that attempt to improve the economic allocation process . . . To the extent that contingent valuation is interpreted as an opinion poll about the environment in general, rather than a measure of preferences about a specific public project, regulators should recognize this concern. However, public policy will do better if expert opinion is used to evaluate  specific projects . . .

Here's my problem with Hausman's answer.  Experts are not unbiased.  They choose their areas of inquiry and expertise based on issues they perceive to be relatively important.  Experts are a non-random sample of the population whose values and judgments are unlikely to mirror the populations'.  Moreover, as this very issue of the Journal of Economic Perspective illustrates, experts often disagree about the meaning of the same set of facts.

Maybe the answer to determining the value of public goods isn't surveys, but while I value expert advice and opinion, I don't think it's a good idea to hand them over the decision making reigns.  The beauty of market-based decisions is that it allows people with competing preferences (defined by their choices) and beliefs to act on their own values and information in a decentralized process that adapts well to change.  Of course, the trouble with goods like the environment is the lack of markets to carry out this allocation process.

Nevertheless, the goal should be to try to find creative mechanisms that simulate what markets do well.  That's one thing I don't like about contingent valuation - it's static and does not allow people to learn and update their beliefs and preferences.  Figuring out how to create new mechanisms and institutions is where I think the future lies - not rehashing a twenty year old debate about contingent valuation.