How surveys can mislead

Beef Magazine recently ran a story about changing consumer attitudes.  The story discussed the results of a nationwide survey which asked the question: "How has your attitude about the following issues changed during the past few years?"  Here is a screenshot showing the results  


So, according to the survey, 29%+35%=64% of consumers are today more concerned about antibiotics than they were a few months ago.  In fact, the figure suggests that more than half of the respondents are more concerned today about antibiotics, hormones, GMOs, animal handling, and farmer values.   

I would submit that these findings are almost entirely a result of the way the question is asked.  Are you more concerned about issue X today?  Well, of course, any reasonable, caring person is today more concerned about X.  Indeed, why would you even be asking me about X unless I should be more concerned?

More generally, drawing inferences from such questions shows the danger of taking a "snapshot" as the truth.  To illustrate, let's compare how the above snapshot looks compared to the trends that come up in the Food Demand Survey (FooDS) I've been conducting for eight months.  

In that survey, I ask over 1,000 consumers each month a question, "How concerned are you that the following pose a health hazard in the food that you eat in the next two weeks?"  where the five-point response scale ranges from "very unconcerned" to "very concerned".  

I pulled out responses to the four issues that most closely match the survey above and plotted the change over time (I created an index where the responses in each month are relative to the response back in May which was set equal to 100).  If people are generally more concerned about these issues today compared to six months ago, it isn't obvious to me from the graph below.

So, a word of caution: you can't take every survey result at face value.  These sorts of comparisons show exactly why our Food Demand Survey is valuable: it replaces a snapshot with a trend.