The dog's blog no. 10: Evidence shows you're right to take a pinch of salt with newspaper reports of health effects
Evidence that you should take a pinch of salt with what you read in the newspapers
Newspapers are always telling us what to do to prevent diseases and live longer. Drink less coffee, take vitamins, or aspirin, or eat certain superfoods to prevent cancer. You’ve probably suspected that you need to be cautious about believing these claims and now there is some evidence in support of taking a skeptical view.
A French team examined how English-language newspapers report associations between diseases (e.g. depression, Parkinson’s disease, breast cancer) and lifestyle (e.g. smoking, eating meat) or other factors (e.g. genetic risk). They found 4,723 studies that were included in 306 meta-analyses* on these associations. They also searched for newspaper reports of the studies and found that 156 of the individual studies and 5 of the meta-analyses had been covered in newspaper articles (1,561 articles in total). They chose not to include reports of studies of treatment effects (including medicines) because the publication and reporting of these are very likely to influenced by stakeholders (often pharmaceutical companies).
It’s the nature of things that the first findings on new scientific questions are later, through further studies and meta-analysis, often contradicted or found to be much smaller effects than appeared at first. But the French study found that journalists appear to be none too keen to report negative effects or to report if earlier findings were later found to be contradicted. They found that:
- new findings were more likely to be covered than later reports on the same findings.
- new findings that reported no link between a disease and a hypothesised risk factor were never covered in newspaper reports.
- later findings that contradicted the findings of an initial publication reporting a link between a disease and a hypothesised risk factor that had received media attention were almost never reported in newspapers.
This shows that there is bias in the way the scientific research is reported in the mass media. As veterinary professionals we need to be careful about where we get our information and this underlines the need for independent sources of information. We also have a role in helping pet owners understand about the uncertainties in scientific knowledge.
*a meta-analysis is a statistical analysis that combines the results of individual studies, to give a summary of the overall size of an effect.
Health scoops are misleading. Rev Prescrire 2017; 37: 776
Dumas-Mallet E et al. Poor replication validity of biomedical association studies reported by newspapers. PLOS ONE 2017; 12: e0172650: 15 pages.