Saturday, January 14, 2017

When doing data reporting, look at the raw numbers, not just at percentages —and write an accurate headline

A headline in The New York Times today reads “In the Shopping Cart of a Food Stamp Household: Lots of Soda.” Is it true?

The story itself provides hints that the headline is misleading, and likely to damage the image of the SNAP program and its beneficiaries. This is dangerous, considering that many readers look at clickbaity headlines, like the NYTimes one, but don't read stories. SNAP households aren't different than the rest of households. Most Americans buy and drink way too much soda and, as a result, obesity and Type II diabetes have reached epidemic levels.

The story says that households that receive food stamps spend 9.3% of their grocery budget on soft drinks, while families in general spend 7.1%. This is one of those cases when reporting just percentages, and not taking into account other variables, such as total spending in groceries, sounds fishy.

Here's why: Never focus just on the data in front of your eyes, or on derived variables, like percentages or rates. Think about the raw numbers behind them. Say that you are comparing two families of four people each, a SNAP one and a non-SNAP one. They spend, respectively, $100 and $200 on groceries —my guess is that SNAP recipients are poorer than Americans in general.

The SNAP household would be spending $9.3 a week on soda, or $2.3 per person; the non-SNAP one —$14,2, or $3.6 per person! Who's buying “lots of soda” now?

To add insult to injury, the story says about the report from the Department of Agriculture: “One limitation of the report was that it could not always distinguish when SNAP households used their benefits, other money or a combination of the two to pay for transactions.” Michele Simon, a lawyer, is quoted in the story: “This is the first time we’ve had confirmation that this massive taxpayer program is promoting all the wrong kinds of foods.” Well, that isn't true. If the story is a good reflection of the U.S.D.A. report, there isn't enough basis to claim that tons of SNAP money is being used to pay for sugary drinks.

This doesn't render the story completely wrong —I couldn't find the original source,— but it does challenge its headline, which singles out SNAP families. The reactionary insurgency taking over government in a few days will likely use this as ammunition to attack safety net programs, as they know that most people share headlines through Facebook and Twitter, but don't spend time reading long and nuanced stories.

(One last note: newspaper reporters often don't write the headlines for their own stories; editors do.)

UPDATE: This post on Facebook by University of Minnesota's Joe Soss provides much more information, and a link to the report itself. I'll take a look at it, as it looks like that the story is much worse than it seemed to me at first.


  1. is the summary report, which I assume is what was used for the article. I and others have pointed out that baby food spending is the biggest difference between the two family types, though as you note, the comparison is virtually meaningless as published.

    1. I forgot to mention that the huge uncertainty implied in the article probably renders the 9.3% vs 7.1% difference meaningless