Chance News 105

A fake study on chocolate

Paul Alper sent a link to the following:

I fooled millions into thinking chocolate helps weight loss. Here's how.

by John Bohannon, io9.com, 27 May 2015

Publishing under the pseudonym Johannes Bohannon, at his own respectably named website for the "Institute of Diet and Health", Bohannon announced the results of a deliberately faulty study designed to show that eating chocolate promotes weight loss. These findings should have sounded too good to be true, but that didn't stop the news and social media outlets from uncritically reporting the findings. The i09.com story linked above includes screen captures from a number of these reports.

The whole story is worth reading for details of how the hoax was conceived and conducted. You can also listen to an NPR interview ("All Things Considered," 29 May 2015). The study used Facebook to recruit a mere 15 volunteers, and randomly assigned them to one of three groups: low carb diet, low carb diet plus one chocolate bar a day, and a control group. Subjects weighed themselves daily, reported on sleep quality and other measures, and had variety of blood work measurements taken, for a total of 18 variables. As Bohannon explains, as long as you don't specify up front what you are looking for, a study with so few subjects and so many variables is bound to turn up a statistically significant result. On io9.com, he calls the design "a recipe for false positives."

Of course, such results will not be reproducible, but the story went viral before anyone asked hard questions. NRP host Robert Siegel wondered if we can take some solace in the fact that reputable sources like the Associated Press or the New York Times did not pick up the story. Bohannon responds that the tabloid press, with much larger readership, had already done the damage.

For additional perspective on all of this, see What junk food can teach us about junk science (NPR, 8 June 2015).

Italian scientists cleared in Aquila case

“Scientists jailed for manslaughter ... cleared”
Daily Mail.com, November 11, 2014

In 2012 seven Italian scientists were sentenced to 6 years each, and fined a total of about $7 million in damages, because of their “misleadingly reassuring statement” to the people of Aquila just prior to a 6.3 earthquake that resulted in the deaths of 309 people.

Some of the scientists defended their actions in a 2013 Scientific American article [1].

In 2014 six of them were cleared of all charges, and the seventh, who had claimed that there was “no danger,” had his sentence reduced to two years.

For more about this case, see Chance News 91 and Chance News 89.

Submitted by Margaret Cibes

Tanzania’s new Statistics Act under fire

The June 2015 issue of Significance magazine contained a brief piece about Tanzania’s new Statistics Act, noting that an up-to-date version of the act was not publicly available at press time. A National Geographic blogger[2] stated that, as of May 13, 2015, there was no official word about whether the president had signed the act.

Nevertheless, a Tanzanian blogger has prepared a 6-page critique of the act -- “The Statistics Act, 2013 – A rapid analysis”, August 4, 2015 [sic] -- in which he discusses his 5 key areas of concern:
1. Uncertainty about who is allowed to generate statistics and what authorization is required
2. Restrictive rules about disseminating survey micro-data
3. Obstacles to whistleblowing without any public interest protections
4. Severe restrictions on the publication or communication of any “contentious” statistical information, and making illegal the publishing/communicating of “false” statistical information or information that “may result in the distortion of facts,” with no protections for acting in good faith
5. Severe penalties for those found guilty of offenses under the bill

For more discussion of potentially problematic aspects of the act, by an English blogger with the adopted Tanzanian name of Mtega[3], see:
“Three (government) statistics that could be illegal under Tanzania’s new Statistics Act”, and
“Four bills later”.

For the act itself, without any 2014 amendments, see: “The Statistics Act, 2013”, United Republic of Tanzania, June 2013.

Submitted by Margaret Cibes