![]()
Is coffee a miracle cure, or is it a huge risk to your health? If you sifted through the recently reported scientific studies on the topic, you may not come away with a very clear conclusion.
According to various media outlets, studies have suggested that coffee may help reverse liver damage, help prevent colon cancer, decrease the risk of endometrial cancer, and increase the risk of miscarriage.
"Coffee today is like God in the Old Testament: It will either save you or kill you depending on how much you believe in its magic powers," Last Week Tonight host John Oliver said on Sunday. "After a certain point, all that ridiculous information can make you wonder, is science bullshit?"
Science is supposed to make us more informed. But as Oliver explained, the way new studies are presented in media often misinforms the general public — and may lead some people to distrust the research.
That perception can cause huge problems: It can lead people to believe that all the science on climate change is wrong, or that vaccines are capable of causing autism, both of which are the wrong conclusions based on the scientific consensus.
"Science is by its nature imperfect, but it is hugely important," Oliver said. "And it deserves better than to be twisted out of proportion and turned into morning show gossip."
There are several reasons why so much bad research seems to come out on a regular basis, Oliver explained:
Journals often will only publish research if it produces striking results, encouraging scientists to tweak their methodology until they can produce an experiment with big findings. (One example is p-hacking, in which scientists collect a bunch of variables to try to find something that's statistically significant, even if the correlation isn't very strong. My colleague Julia Belluz wrote more about that here.)
Replication studies, science's best way of verifying certain findings, are almost never done. As Elizabeth Iorns of Science Exchange explained, "Replication studies are so rarely funded, and they're so underappreciated. They never get published. No one wants to do them; there's no reward system there in place that enables it to happen."
Press releases, even by prestigious schools, often exaggerate study results to make them sound sexier. "It's like a game of telephone," Oliver said. "The substance gets distorted at every step." (One example came in January when the University of Maryland greatly exaggerated findings in some chocolate milk research.)
The media, in turn, often blows findings out of proportion. Reporters very rarely go through a study's methodology or explain the caveats, such as a small sample size. For example, one widely reported study from last year found driving while dehydrated was just as bad as driving drunk. But the trial involved just 12 men — a tiny sample size, yet one that was rarely mentioned in news reports.
The result is a less informed public — and one that may mistrust science. This was best exemplified recently on the Today show, when co-host Al Roker said, "I think the way to live your life is you find the study that sounds best to you and go with that."
Oliver's response was dead-on: "No! No, no, no, no, no, no, no, no! In science, you don't just get to cherry-pick the parts that justify what you were going to do anyway. That's religion. You're thinking of religion."
Science is often flawed. It's time we embrace that.
This is why you shouldn't believe that exciting new medical study.
An unhealthy obsession with p-values is ruining science.
What psychology’s crisis means for the future of science: The field is currently undergoing a painful period of introspection. It will emerge stronger than before.
