Here’s a great lesson in information literacy relating to the ways that health research is reported in the news media. This originates with a study in the Journal of American Medical Association, A Guide to Reading Health Care News Stories by Gary Schwitzer. Here’s some notes from it highlighting reporting problems (and an On the Media Story below about it). When you read about health news, you can use as rules of thumb to catch any potential reporting problems.
“Risk Reduction Stated in Relative, Not Absolute, Terms”
A 50% improvement may mean going from 2 out of 100 people to 1 out of 100 people. Yes, 50% but not really a huge improvement.
“Failure to Explain the Limits of Observational Studies”
Observational studies are types of studies where people who take one action (eat more apples) have a particular result (have less heart attacks). The major limit is that the action may NOT actually cause the result. These studies can be useful as a first step, but more research is often needed to REALLY understand what is happening
“The Tyranny of the Anecdote”
Anecdotes are single stories used to illustrate a point. Often, these stories are not representative of all experiences. They often ignore individuals who drop out of studies. Reporters must give appropriate context.
“Single-Source Stories and Journalism Through News Releases”
Any time a story uses only one source (like one study) this is a problem because scientists never rely on just one study. This becomes more of a problem (biased info) when the reporter is relying on a press release from a drug manufacturer or other company.
“Stories About Screening Tests That Do Not Explain the Tradeoffs of Benefits and Harms”
There are instances where screenings can find false positives (a test that finds a disease in someone who does NOT have the disease). False positives can lead to addition (often more invasive) tests. News stories that discuss disease screenings should include harms of these screenings.
“Fawning Coverage of New Technologies”
We like flashy new technologies, but often, the new technologies are only slightly better (or not better) than existing (less flashy) treatments. We should be wary about stories that go crazy over the glitzy new approach without comparing it to existing approaches.
Here is the story for On the Media about this report:
THE WORRIED WELL WHIPPED INTO A FRENZY