![]()
I’m not sure what the University of Maryland (UMD) is waiting for. The sharks are circling. The longer UMD waits to definitively retract their now-infamous “chocolate milk helps athletes with concussion” study, the guiltier they look.
Andrew Holtz of HealthNewsReview.org) pressured UMD to release details of the unpublished study, things got more ugly. And as wreckage accumulated, I couldn’t take my eyes off.
Yesterday, Jesse Singal of New York magazine published key details including a Power Point file, provided by UMD, in which the university listed major aspects of the study. The file is 43 slides long, but I can sum up the results very concisely …
The UMD study found exactly what you would expect to find if Fifth Quarter Fresh had zero effect on players with concussions.
The math is pretty simple. The researchers decided that a result was “statistically significant” if its p-value was less than 0.10. That means that if Fifth Quarter Fresh has zero effect, then there is still a 10% chance of finding a “significant” result by pure luck. Subsequently, the researchers tested 36 different things. If you test 36 things on a product with zero effect, and you have a 10% chance of “significance” due to pure luck each time, then you can expect 4 things to be “significant” (rounding off 3.6) due entirely to luck.
Guess how many “statistically significant” results the UMD researchers found among players with concussions? Four.
So, ironically, if the study’s goal were to prove that Fifth Quarter Fresh does not reduce the effects of concussions, then I’d say they did a damn good job. Yet the University of Maryland reported the exact opposite … about a deeply serious public health problem that even the powerful National Football League is petrified of.
And we haven’t even talked sample size yet. The study only had 7 players with “usable” data who suffered a concussion and drank Fifth Quarter Fresh. It doesn’t matter what is “statistically significant” – as I said to Casey Hinds in Beyond Chron yesterday, it’s irresponsible to make decisions about players’ safety based on how 7 kids responded to a product.
Furthermore, I’m focusing on the statistics because journalists and other experts have dissected the long list of other scientific and ethical problems with the study:
A state university using corporate money to test a specific corporate product
Designing a study that didn’t actually test the corporate product. For all we know, Fifth Quarter Fresh might be no better than a Starbucks mocha frappuccino.
Issuing a press release when they had no specific results to share, nothing had been peer-reviewed, and the lead researcher was unavailable
Claiming that the university Institutional Review Board approved of not obtaining parental permission to collect concussion-related data from teenagers – data that were being collected by non-university employees (high school athletic departments)
Not to mention pitting one public health problem (concussions) against another (obesity, diabetes, etc.) It’s essentially capitalizing on parents’ anxiety about one health problem to promote a chocolate milk product that has more sugar than a 12-oz can of Coke. What’s their follow-up study? Testing the effect of shoulder pads coated with asbestos?
To be fair, UMD claims to be conducting an “institutional review” of the study. Yet they continue to post the press release and are oddly comfortable with Fifth Quarter Fresh trumpeting the results on its website. UMD also hasn’t answered the biggest ethical questions, particularly regarding not obtaining parental permission (to my knowledge they haven’t answered it.)
I never thought I’d say this about anyone – but UMD might want a lesson from Coca-Cola on transparency. Chocolate stains keep emerging, and the more UMD hides, the more curious and outraged people become. This story isn’t going anywhere. UMD needs to openly admit where things went wrong and take definitive action to correct them.
Silver lining to this story
This story was science at its worst but showcased a lot of excellent science journalism. The initial UMD press release didn’t generate much traction, but when journalists and other concerned citizens started poking around and speaking up, the story flared up in the way that it deserved. Thank you to Andrew Holtz, Jesse Singal, Casey Hinds, Sarah Gantz of the Baltimore Business Journal, Julia Belluz of Vox, and others who pressured UMD.
Scientists often gripe about journalists misrepresenting their work. I think scientists need to do a better job of representing their own work. This was an exemplary case of journalists finding the truth even if scientists stood in their way.
(Image Sources: Flickr/ehpien and Flickr/Kiwanis Club of Lafayette under a Creative Commons license; no changes were made)
I am an Assistant Professor at the University of Texas Health Science Center at Houston – Austin Regional Campus, where I specialize in childhood obesity policy research and systems science. This is my personal blog; any views or opinions expressed do not represent the University of Texas (or anyone else with power.)
My research has been featured in the New York Times, NPR, CNN, Wall Street Journal, and several other media outlets. You can view my website, drdantaber.com, or follow me on Twitter at @DanTaber47, where I often tweet about obesity news, school nutrition, public transit, systems science, and occasionally random topics like Seattle sports, marathon training, or my latest obscure vacation destination.
