The danger of making public policy based on epidemiological studies

Scientific American recently ran an interesting story on antioxidants.  For a while, it seems, experts promoted antioxidants based on epidemiological studies that seemed to suggest they increased longevity.  It is a good thing these experts didn't convince policy makers to subsidize or mandated more vitamins and antioxidants in food years ago (although we do have mandated vitamin D milk and iodine in salt), only to discover this:

Vitamins Kill Epidemiological studies show that people who eat lots of fruits and vegetables, which are rich in vitamins and other antioxidants, tend to live longer and are less likely to develop cancer compared with those who do not.  So it seemed obvious that supplementing diet with antioxidants should lead to better health.  But the results of the most rigorously designed studies do not support that assumption.  Indeed, the evidence shows that some people who take certain supplements are actually more likely to develop life-threatening illnesses, such as lung cancer and heart disease.

There are many epidemiological studies showing correlations across people in the intake of one food (e.g., meat, chocolate, blueberries, wine) and some undesirable or desirable health outcome (e.g., cancer, heart disease, longevity, etc.).  But, it cannot be repeated enough: correlation is not causation.    

Do People Want More Food Regulation? Or Less?

Over at Reason.com, Baylen Linnekin reports on the results of a recent poll by the Associated Press-NORC Center for Public Affairs Research.  According to Linnekin, the poll shows little public support for food taxes and bans.  Balyen contrasts the recent survey with some previous survey work I’d done which seems to show the opposite.

A vast literature on polling and survey research shows that subtle changes in wording and response categories can result in large shifts in behavior.  Thus, it is useful to compare the two questions side-by-side. In the end, I think you’ll find much more similarity in the two studies than perhaps first meets the eye.

Here is the exact AP-NORC poll question and response categories (it was a telephone poll and you can find the script here):

Do you favor, oppose, or neither favor nor oppose the following government policies?FOR EACH FAVOR OR OPPOSE: Is that strongly (favor/oppose) or somewhat (favor/oppose)?

Requiring more physical activity in schools (84%, 89%)

Providing nutritional guidelines and information to people about how to make healthy choices about diet and exercise (83%, 90%)

Funding farmers markets, bike paths and other healthy alternatives (74%, 81%)

Providing incentives to the food industry to produce healthier foods (73%, 80%)

Requiring restaurants to post calorie information on menus (70%, 78%)

Banning advertisements for unhealthy foods aimed at children (44%, 53%)

Placing a tax on the sale of unhealthy foods and drinks (31%, 40%)

Limiting the types or amounts of foods and drinks people can buy (15%, 25%)

As shown above, there were eight issues listed (in random order across respondents).  I’ve listed them in order of support.  I’ve also listed the % favoring in parentheses beside each issue, then a comma and the % favoring plus not opposed (to which I’ve added in the “neither opposed nor unopposed” to the total).

I’d hardly call this set of responses free market or libertarian.  There is ample support for requirements, subsidies, and mandates.  Given the way the question was asked, I could see a respondent perceiving the question to ask something like “rank these interventions from most favored to least favored.”  It would be interesting to know if there were strong order effects.  For example, if “taxes” came first, were they more/less supported than if they came last.  In any event, there is apparently weaker support (and less than majority support) for “fat taxes” and bans on amounts or types of foods people can buy (although, my gut feel is that if you replaced the vague “types or amounts of foods” with something specific like “transfats” or “GMOs” you might get a very different answer)

My study (published in Food Policy) phrased the questions a different way and used an online format.  I asked about preference for government action related to 10 food issues.  None of them match up perfectly with the list of eight above, but I’ll pull out two that are somewhat similar to the above. 

Each question asked:

Which of the following best describes your view on what the U.S. government should do?

Each question had two options that involved more government action, a status-quo option, and two options that involved less government action. 

Here are the results from one question about healthy food with % of respondents falling into each category:

Which of the following best describes your view on what the U.S. government should do?

Ban the use of transfats, saturated fats, and other unhealthy ingredients in food production (15.1%)

Increase regulations to restrict the use of transfats, saturated fats, and other unhealthy ingredients in food production (38.8%)

Maintain current policies on transfats and saturated fats (e.g., mandatory labeling in the supermarket)       (31.6%)

Reduce regulations on transfats and saturated fats    (2.7%)

Make no law regarding transfats, saturated fats, and other unhealthy food ingredients, leaving people to take responsibility for their own diet          (11.8%)

So, 53.9% wanted more regulation on this topic, 31.5% wanted the status-quo and 14.5% wanted less regulation.

Here are the results from another question I asked:

Which of the following best describes your view on what the U.S. government should do?

Create an agency to plan food production and distribution to improve nutritional intake (15.4%)

Use extensive taxes and subsidies to promote healthier foods           (14.2%)          

Maintain current regulations designed to promote healthier foods which include mandatory nutritional labels on foods and establishing suggested dietary intake (53.1%)     

Decrease efforts to promote healthier foods  (5.3%)

Eliminate all food health regulations; allow citizens to make their own food choices (11.9%)        

So, 29.7% wanted more regulation on this topic, 53.1% wanted the status quo, and 17.2% wanted less regulation.

In total, seven of the questions I asked about garnered majority support for government action and the most favorable related to issues that could be perceived as relating to food safety, food affordability, and animal welfare. Three issues did not garner support for more government action.  So, in my study 70% of the issues raised were such that people wanted more government action compared to the status quo or less government action. 

The AP-NORC poll asked about eight issues, and (depending on how you treat the “undecideds”), either 62.5% or 75% garnered majority support for more government action. 

So, yes, we can find a couple questions were we “free market” folks can take a bit of comfort.  However, the overall response patterns in both surveys are much more statist than I am comfortable with.  That’s one reason I decided to write The Food Police (you can also read more on my interpretation of these results here)  I’m hopeful I can bring more folks over to my way of thinking by presenting a perspective that differs from the one normally offered by many food writers.   

Do Food Consumers Vote Differently Than They Shop?

According to some research I just published in the Journal of Agricultural and Resource Economics with a former graduate student, Kate Brooks, who is now an assistant professor at the University of Nebraska, the answer is "yes."  

Our research suggests caution in using people's shopping behavior (as, for example, indicated by grocery story scanner data) to infer which public policies they may or may not support.  In the particular application we studied, people were willing to pay large premiums to avoid milk and meat from cloned cows when asked what they wished to buy when shopping in a grocery store.  One might conjecture from this behavior, then, that the consumers would approve of a government ban on use of clones in meat and milk production.  

According to our research, that conjecture would be wrong.  The majority of consumers did NOT favor a ban on cloning in food animals.  In fact, most people would demand compensation if a ban were enacted (rather than be willing to pay to have the ban).  This finding defies many of the explanations often given for differences in voting and shopping behavior, such as the consumer-vs-citizen hypothesis or the hypothesis that consumers perceive the existence of externalities.  The behavior is more consistent with the notion that people have an option value (they don't want to get rid of a technology that may produce some promising result in the future even if they don't want it now) or that people respect the freedom of others to arrive at their own choices even if they happen to be at odds with one's own preferences.     

The other interesting thing about our finding is that is exactly the opposite of what has been observed in other food issues.  For example, in California, 63.5% of voters voted in favor of Prop 2 in 2008 to effectively ban battery cages in egg production.  Yet, the retail market share of cage free eggs is less than 5%.  In this case, shoppers aren't willing to shell out the extra bucks for cage free eggs in the grocery store, but they enthusiastically voted to ban the product they normally buy in the voting booth.  Why?  Hard to say.  My feeling is that the costs are much more salient in the store than in the voting booth.  Another possibility is that the universe of voters is different than the universe of shoppers (all voters shop but not all shoppers vote).  There are, of course, other possible explanations.  

Gaining a better understanding why people behave differently when shopping and voting is a key area of future research for food economists.  And, the fact that people often behave so differently in the two environments represents a key challenge for food economists who conduct regulatory cost-benefit analysis and advise policy makers.  

How Reliable are Surveys of Public Opinion and Preference?

A day after the presidential election, it is useful to evaluate the usefulness of the surveys and polls that were used to predict election outcomes.  By and large, my assessment is that most of the polls got it about right.  Republicans complained a lot about state-level polls in Ohio and elsewhere saying that Democrats were being weighted too heavily, but in the end, it appears the pollsters had it right.  

Now turn to economics, where the latest issue of the Journal of Economic Perspectives just arrived in my inbox.  In the issue are three articles on the use of "contingent valuation", which is a survey method used to ask people how much they are willing to pay for some good - normally a public good (like the environment) that is not traded in a market setting.  The first two articles by Kling, Phaneuf, and Zhao and by Carson (here and here) are generally pro-contingent valuation, the last by Jerry Hausman at MIT is against.   

I've spent a lot of time studying contingent valuation and other consumer research methods.  The methods are far from perfect and they suffer from many well-known biases.  At the end of the day, however, my question is: what is the alternative?  

Here is Hausman's answer (footnote and references removed):

I am often asked what should be done given my view that contingent valuation should not be used. Should nonuse value be ignored? My view is that expert government agencies and Congress should make informed decisions and enact regulations that attempt to improve the economic allocation process . . . To the extent that contingent valuation is interpreted as an opinion poll about the environment in general, rather than a measure of preferences about a specific public project, regulators should recognize this concern. However, public policy will do better if expert opinion is used to evaluate  specific projects . . .

Here's my problem with Hausman's answer.  Experts are not unbiased.  They choose their areas of inquiry and expertise based on issues they perceive to be relatively important.  Experts are a non-random sample of the population whose values and judgments are unlikely to mirror the populations'.  Moreover, as this very issue of the Journal of Economic Perspective illustrates, experts often disagree about the meaning of the same set of facts.

Maybe the answer to determining the value of public goods isn't surveys, but while I value expert advice and opinion, I don't think it's a good idea to hand them over the decision making reigns.  The beauty of market-based decisions is that it allows people with competing preferences (defined by their choices) and beliefs to act on their own values and information in a decentralized process that adapts well to change.  Of course, the trouble with goods like the environment is the lack of markets to carry out this allocation process.

Nevertheless, the goal should be to try to find creative mechanisms that simulate what markets do well.  That's one thing I don't like about contingent valuation - it's static and does not allow people to learn and update their beliefs and preferences.  Figuring out how to create new mechanisms and institutions is where I think the future lies - not rehashing a twenty year old debate about contingent valuation.