Chance News 105: Difference between revisions
| Line 11: | Line 11: | ||
Publishing under the pseudonym Johannes Bohannon, at his own respectably named website for the "Institute of Diet and Health", Bohannon [http://instituteofdiet.com/2015/03/29/international-press-release-slim-by-chocolate/ announced the results] of a deliberately faulty study designed to show that eating chocolate promotes weight loss. These findings should have sounded too good to be true, but that didn't stop the news and social media outlets from uncritically reporting the findings. The i09.com story linked above includes screen captures from a number of these reports. | Publishing under the pseudonym Johannes Bohannon, at his own respectably named website for the "Institute of Diet and Health", Bohannon [http://instituteofdiet.com/2015/03/29/international-press-release-slim-by-chocolate/ announced the results] of a deliberately faulty study designed to show that eating chocolate promotes weight loss. These findings should have sounded too good to be true, but that didn't stop the news and social media outlets from uncritically reporting the findings. The i09.com story linked above includes screen captures from a number of these reports. | ||
The whole story is worth reading for details of how the hoax was conceived and conducted. You can also listen to [http://www.npr.org/sections/thesalt/2015/05/29/410609184/trickster-journalist-explains-why-he-duped-the-media-on-chocolate-study an NPR interview] ("All Things Considered," 29 May 2015). The study used Facebook to recruit a mere 15 volunteers, and randomly assigned them to one of three groups: low carb diet, low carb diet plus a daily chocolate bar, and a control group. Subjects weighed themselves daily, reported on sleep quality and other measures, and had variety of measures taken from blood work, for a total of 18 variables. As Bohannon explains, as long as you don't specify up front what you are looking for, a study with so few subjects and so many variables is bound to turn up a statistically significant result. On | The whole story is worth reading for details of how the hoax was conceived and conducted. You can also listen to [http://www.npr.org/sections/thesalt/2015/05/29/410609184/trickster-journalist-explains-why-he-duped-the-media-on-chocolate-study an NPR interview] ("All Things Considered," 29 May 2015). The study used Facebook to recruit a mere 15 volunteers, and randomly assigned them to one of three groups: low carb diet, low carb diet plus a daily chocolate bar, and a control group. Subjects weighed themselves daily, reported on sleep quality and other measures, and had variety of measures taken from blood work, for a total of 18 variables. As Bohannon explains, as long as you don't specify up front what you are looking for, a study with so few subjects and so many variables is bound to turn up a statistically significant result. On io9.com, he calls the design "a recipe for false positives." | ||
Of course, such results will not be reproducible, but the story went viral before anyone asked hard questions. NRP host Robert Siegel wondered if we can take some solace in the fact that reputable sources like the Associated Press or the ''New York Times'' did not pick up the story. Bohannon responds that the tabloid press, with much larger readership, had already done the damage. | Of course, such results will not be reproducible, but the story went viral before anyone asked hard questions. NRP host Robert Siegel wondered if we can take some solace in the fact that reputable sources like the Associated Press or the ''New York Times'' did not pick up the story. Bohannon responds that the tabloid press, with much larger readership, had already done the damage. | ||
Revision as of 14:37, 9 June 2015
Quotations
Forsooth
A fake study on chocolate
Paul Alper sent a link to the following:
- I fooled millions into thinking chocolate helps weight loss. Here's how.
- by John Bohannon, io9.com, 27 May 2015
Publishing under the pseudonym Johannes Bohannon, at his own respectably named website for the "Institute of Diet and Health", Bohannon announced the results of a deliberately faulty study designed to show that eating chocolate promotes weight loss. These findings should have sounded too good to be true, but that didn't stop the news and social media outlets from uncritically reporting the findings. The i09.com story linked above includes screen captures from a number of these reports.
The whole story is worth reading for details of how the hoax was conceived and conducted. You can also listen to an NPR interview ("All Things Considered," 29 May 2015). The study used Facebook to recruit a mere 15 volunteers, and randomly assigned them to one of three groups: low carb diet, low carb diet plus a daily chocolate bar, and a control group. Subjects weighed themselves daily, reported on sleep quality and other measures, and had variety of measures taken from blood work, for a total of 18 variables. As Bohannon explains, as long as you don't specify up front what you are looking for, a study with so few subjects and so many variables is bound to turn up a statistically significant result. On io9.com, he calls the design "a recipe for false positives."
Of course, such results will not be reproducible, but the story went viral before anyone asked hard questions. NRP host Robert Siegel wondered if we can take some solace in the fact that reputable sources like the Associated Press or the New York Times did not pick up the story. Bohannon responds that the tabloid press, with much larger readership, had already done the damage.
For additional perspective on all of this, see What junk food can teach us about junk science (NPR, 8 June 2015).