Do Survey Respondents Pay Attention?

Imagine taking a survey that had the following question. How would you answer?

If you answered anything but "None of the Above", I caught you in a trap.  You were being inattentive.  If you read the question carefully, the text explicitly asks the respondent to check "None of the Above."  

Does it matter whether survey-takers are inattentive?  First, note surveys are used all the time to inform us on a wide variety of issues from who is most likely to be the next US president to whether people want mandatory GMO labels.  How reliable are these estimates if people aren't paying attention to the questions we're asking?  If people aren't paying attention, perhaps its no wonder they tell us things like that they want mandatory labels on food with DNA.

The survey-takers aren't necessarily to blame.  They're acting rationally.  They have an opportunity cost of time, and time spent taking a survey is time not making money or doing something else enjoyable (like reading this post!).  Particularly in online surveys, where people are paid when they complete the survey, the incentive is to finish - not necessarily to pay 100% attention to every question.

In a new working paper with Trey Malone, we sought to figure whether missing a "long" trap question like the one above or missing "short" trap questions influence the willingness-to-pay estimates we get from surveys.  Our longer traps "catch" a whopping 25%-37% of the respondents; shorter traps catch 5%-20% depending on whether they're in a list or in isolation.  In addition, Trey had the idea of going beyond the simple trap question and prompting people if they got it wrong.  If you've been caught in our trap, we'll let you out, and hopefully we'll find better survey responses.  

Here's the paper abstract.

“This article uses “trap questions” to identify inattentive survey participants. In the context of a choice experiment, inattentiveness is shown to significantly influence willingness-to-pay estimates and error variance. In Study 1, we compare results from choice experiments for meat products including three different trap questions, and we find participants who miss trap questions have higher willingness-to-pay estimates and higher variance; we also find one trap question is much more likely to “catch” respondents than another. Whereas other research concludes with a discussion of the consequences of participant inattention, in Study 2, we introduce a new method to help solve the inattentive problem. We provide feedback to respondents who miss trap questions before a choice experiment on beer choice. That is, we notify incorrect participants of their inattentive, incorrect answer and give them the opportunity to revise their response. We find that this notification significantly alters responses compared to a control group, and conclude that this simple approach can increase participant attention. Overall, this study highlights the problem of inattentiveness in surveys, and we show that a simple corrective has the potential to improve data quality. ”

The easy-going hunter-gather life. Or not.

How many times have you read something like the following?

“Rather than heralding a new era of easy living, the Agricultural Revolution left farmers with lives generally more difficult and less satisfying than those of foragers. Hunter-gatherers spent their time in more stimulating and varied ways, and were less in danger of starvation and disease. . . The average farmer worked harder than the average forager, and got a worse diet in return. The Agricultural Revolution was history’s biggest fraud.”

I've read variations on that theme so often, I assumed it must be (at least partially) true during the early stages of the agricultural revolution.  (The above quote is from a recent, bestselling book by Yuval Noah Harari)

In a post a few days ago, Rachael Laudan makes a compelling case against this conventional wisdom.  In short she challenges the way anthropologists and other researchers classified "work" for hunger-gatherers vs. agriculturalists.  It seems, these researchers didn't take food processing into account.

She re-visits the supposedly lax lives of hunter-gathers, focusing in the following passage on what it takes to turn collected nuts into food (i.e., food processing).   

“8 hours a week spent cracking mongongo nuts (minimum, not clear if this includes preliminary cooking to soften the shell or subsequent time extracting the kernel), p. 270

4-7 hours a week spent making and repairing tools (35-64 mins daily), p.277.

15-22 hours a week spent on butchery, meat cooking, and fuel collection (2.2-3.2) p. 278 (no mention of carrying water in ostrich shells)

In short, between 27 and 37 hours a week spent by each adult on food processing and ancillary activities: fuel, water and tools.

Assume an eight-hour work day. Therefore: 3.5-4.5 days a week spent on food processing, tools, fuel, and water.

That is, given any reasonable sense of work, bushmen spent more time dealing with the food they collected than collecting it.

As so often, food processing for humans takes longer than food production or collection.

And the total work week for the bushmen on the lowest of estimates turns out to be between 6 and 7 eight-hour days (not counting child care).”

There's a lot more at the original post.

It's sometimes said that sacred cows make the tastiest burgers.  

Why aren't all chickens cage free?

Jennifer Chaussee has a piece at Wired on hen housing.  She attempts to answer some questions that I get frequently asked: why isn't the egg industry converting faster to cage free?  Isn't that the trend?  Why aren't we all cage free yet?

She writes

“As it turns out, going cage-free requires much more planning, money, and logistical engineering than the seemingly simple notion of setting some hens free would suggest. Ironically, this massive supply chain overhaul stems from consumer demand to return to the egg-producing practices of our pre-industrial past, but without undoing all the positive benefits of scale, affordability, and safety that were achieved through industrialization. It actually took farmers a really long time to figure out how to put the bird in the cage—and it’s going to take a while to figure out how to get it back out.”

Overall, it's a pretty good piece, and recounts many of the issues we've written about in other places.

When will Chipotle recover?

By now, I suspect everyone is well aware of the fall-out from Chipotle's foodborne illness outbreaks.  While I've previously discussed some aspects of the outbreaks, I want to touch on a different dimension here.  What are the financial consequences and when will Chipotle recover?

First off, it is almost impossible to answer the "when will Chipotle recover" with any degree of certainty.  I've seen several stories on impending lawsuits, and the timing and outcome of those legal disputes are somewhat erratic and hard to predict.  

In any event, let's look at what's happened thus far.  Chipotle started having some outbreaks in late summer and early fall, but when the the CDC began reporting outbreaks associated with Chipotle in early November 2015, that's when things started heading south.  Since the middle of October, Chipotle's stock (symbol: CMG) price has fallen by about 40% (from above $700/share to low-to-mid $400/share).  The overall stock market has been tanking in recent weeks, but as the chart below shows, Chipotle's stock (the solid black line) fell far more than did the S&P 500 (the light purple line).

One of the things this result illustrates is the private incentive for companies to invest in food safety.  Here's a little snippet from my forthcoming book, Unnaturally Delicious, on that topic:

“The reputation conveyed by brand names might allow firms to make a bit more money, but it also exposes them to large potential losses in the event of a product recall or food safety event. Reputation is a two way street, and a once solid name can quickly work against you if it becomes tied to bad news. Research shows that meat recalls by publically traded companies typically result in a 1.5% to 3% loss in shareholder wealth. For branded products like hot dogs, a food safety recall tends to reduce sales by more than 20%, and the negative effects persist for more than four months. ”

The statistics cited in the above paragraph come from a couple academic papers by Michael Thomsen and colleagues (see here and here).  Here's another interesting paper from a group of agricultural economists showing that consumers exposed to information about a food safety outbreak reduced their willingness-to-pay for the affected brand up to 50 days after they received the information.   They conclude:

“Results from this study indicated that consumers are willing to change their purchasing behaviors to avoid unsafe products. Both positive and negative information had an effect on consumers’ WTP. Consumers were willing to pay less for the leading-brand chicken after they received negative food safety information compared to a control group that did not receive this information. Participants that received positive food safety information about Ranger brand chicken were consistently willing to pay more for this safer option than for leading-brand chicken. This suggests that when the information about which brands are safer is available, consumers are willing to alter their purchasing behavior to favor the safer alternative, even if it was a relatively unknown brand. Both of these effects appeared to last well beyond the initial exposure to media information.”

So, the length of time it takes to rebound depends, in part, on what sorts of additional positive and negative information come out about Chipotle.  It may also be useful to look at similar events for other companies.  Taco Bell had a widely publicized Salmonella outbreak a few years back, and it really hit the news around the 1st of February 2012.  Here's a plot of the stock price of YUM! Brands, which owns Taco Bell.

You can see a downward movement in mid 2012, but the price rebounded by the end of the year (before falling again). But, the mid-2010 fall was less than 10%.  This helps illustrate the fact that it's hard to generalize.  How the public responds to a recall depends on how they view the company and how the company responds, among other factors.

It's also useful to take a step back and take a longer view.  Yes, the price of Chipotle has fallen about 40% in the past few months.  But, if you'd bought their stock back in 2008, when the price was around $50/share, you'd still be up over 700% after the recalls.  

11 things to know about GMOs

Over the past year, I'd received a large number of inquiries about GMOs.  Some of the questions were from moms, others from farmers, and sometimes from the media.  It seems a common set of questions continued to come up, so I got together with my colleagues Eric and Cheryl Devuyst to put to together this University Fact Sheet to provide some succinct answers.

We answer the following 11 questions as best we can in a mere two pages.

• What is a GMO?
• Why are crops genetically modified?
• Are GMOs safe to eat?
• What crops in the U.S. are genetically modified?
• What are the environmental effects of GMOs?
• Do farmers need to use more pesticides with GMOs?
• How are GMOs regulated?
• Are GMOs banned in Europe?
• Should food companies be required to label foods with GMOs?
• What are the economic effects of farmers using GMOs?
• What are the potential downsides of GMOs?

You can find our answers here.