Sunday, August 18, 2013

When writing about data, don't forget the context —or your own infographics

I was checking my RSS and Twitter feeds a few minutes ago and saw a link to an article at the Poynter Institute's website. This is its headline:

Holy alarming survey, Batman! More than a quarter is a lot, isn't it? Well, it turns out that we don't know. Why not? Because we lack enough context. There isn't a comparison with other majors, as a reader has pointed out. Moreover, the headline is accurate, but incomplete. A more adequate one, in my opinion, would have been: "The percentage of journalism grads who wish they'd chosen another career has remained stable for the past decade and a half." Not that catchy, I'll give you that, but it's more truthful. Here's the evidence, a graph showcased in the article:

Not that worrying, right? In fact my guess is that the slight variation in the time-series chart is due to simple random noise. Here's a a line that may support this conjecture (page 13 in the study):
"Sample error for the 2012 undergraduate data is 2.3%. Sample error terms for earlier surveys ranged from 1.8% (2004) to 3.7% (1988)."
Right after I tweeted about all the above, I double-checked the quotes extracted from the study. Here's a paragraph included in Poynter's story:
"“That it is a significant percent cannot be doubted,” the study’s authors write. "No standard from other fields exists for this question, however, and it seems likely that some graduates would be unhappy with their career choice regardless of which one they had selected. The match between expectation and actual job prospects is unlikely to ever be perfect. One in 20 of the journalism and mass communication graduates each year indicates that she or he had selected the field without ever intending to go into it."
But here's the line that precedes those in the study, which should have been there, too:
"(...) In 2012, about one in four of the graduates responded to one of those questions by saying that they regretted their career choice. The figure is relatively unchanged from a year earlier and about at the average across the 14 years the question has been asked."
In the past few years, I've grown quite concerned about how we, journalists and designers, deal with data, science, and evidence, and about how we explain them to readers. Examples like this one lead me to think that we still have a lot to learn. Why would you highlight the fact that around 25-30% of J-grads regret choosing journalism as a career? Wouldn't it be more newsworthy to report that this number has barely changed in spite of the deep crisis news media —and journalism itself, for that matter— is going through? That's the real story, I believe.