Chance News 70
Quotations
"The interpretation of statistical significance tests is liable to a misconception known as the fallacy of the transposed conditional. In this fallacy, the probability of the data given a hypothesis (e.g., P(D|H), such as the probability of someone being dead given that they were lynched, a probability that is close to 1) is confused with the probability of the hypothesis given the data (e.g., P(H|D), such as the probability that someone was lynched given that they are dead, a probability that is close to zero)."
Why psychologists must change the way they analyze their data: The case of Psi
The authors are criticizing naive use of p-value in the recent ESP research controversy.
Submitted by Paul Alper
”If you averaged the highs and lows of a rollercoaster, it wouldn’t be much of a thrill ride; same with the threat of flu.”
The CDC counts of flu deaths have ranged from 3,300 to almost 50,000 in various years, which makes the average number of deaths per year somewhat unrepresentative of the actual number of deaths:
STATS, September 3, 2010
Submitted by Margaret Cibes
Forsooth
From a novel, The Kills, by Linda Fairstein, Scribner, 2004:
“Hey, how many people do you need to have in a room to guarantee the chance that at least two of them would have the same birthday?”
“I don’t know. Three hundred sixty-four.”
“Hah! Twenty-three. At least two out of every twenty-three people will have exactly the same birthday. Statistical odds. A lot of life is coincidence.”
Submitted by Margaret Cibes
New ESP study raises ruckus
Read about a new study in which a Cornell psychologist claims to have verified "ESP":
“ESP Study Gets Published in Scientific Journal, by Ned Potter, ABC World News, January 6, 2011 (including 2-min video interview).
“Journal’s paper on ESP Expected to Prompt Outrage”, by Benedict Carey, The New York Times, January 5, 2011.
Read the study:
“Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect”, by Daryl J. Bem, Cornell University, Journal of Personality and Social Psychology, 2010.
Read a rebuttal:
“Why Psychologists Must Change the Way They Analyze Their Data”, by Eric-Jan Wagenmakers et al., University of Amsterdam.
We reanalyze Bem’s data using a default Bayesian t-test and show that the evidence for psi ["ESP"] is weak to nonexistent. …. We conclude that Bem’s p-values do not indicate evidence in favor of precognition; instead, they indicate that experimental psychologists need to change the way they conduct their experiments and analyze their data.
For more discussion, see Andrew Gelman's blog (1/6/11):
That silly ESP paper and some silliness in a rebuttal as well.
Submitted by Margaret Cibes based on an ISOSTAT posting by Randall Pruim
Placebos without deception
Patients in study who knew they were taking placebo still felt better
by Deborah Kotz, Boston Globe, 23 December 2010
Although somewhat of an exaggeration, before the invention of aspirin, MDs had nothing to offer except placebos. Today, no study in the medical field can be taken seriously without a control for the so-called placebo effect. In an unusual twist whereby the placebo is the treatment, Kaptchuk, et al openly beforehand informed 37 patients suffering from irritable bowl syndrome (IBS) that they were receiving a placebo “without any medication in it.” The other 43 patients were a control, i.e., no treatment, in that they received no medication whatsoever.
The Globe article put it this way:
The researchers got some astounding results when they gave placebos — gelatin capsules filled with nondigestible cellulose — to patients suffering from irritable bowel syndrome for three weeks. Nearly 60 percent reported an improvement in their symptoms compared with 35 percent of the patients who took nothing beyond their usual treatments. But here is the kicker: The placebo takers knew they were popping the equivalent of sugar pills, yet they still said they experienced less abdominal pain, constipation, or loose stools during the study.
Discussion
1. Here is the table of treatment outcomes from the Kaptchuk paper and an associated graphic (the phrase “Open Placebo” means no deception). Use a statistics package to verify the p-values given there.
2. Just to confuse things now that the power of the placebo has been firmly established for IBS, consider this which also was recently published:
In two studies involving more than 1,200 subjects with diarrhea-predominant IBS, researchers found that a two-week course of the antibiotic rifaximin helped relieve symptoms not only during treatment, but also for weeks after the medication was stopped.
Participants who were randomly assigned to receive the drug reported less bloating and abdominal pain, and improved stool consistency for up to 10 weeks, say the authors, whose paper is published in this week's New England Journal of Medicine.
About 40 per cent or more of subjects given the thrice-daily rifaximin pill had significantly diminished IBS symptoms compared to those given placebo, or dummy pills, the study found.
Dr. Lawrence Cohen who was not involved with the study
said rifaximin could have potential, although how much is difficult to determine because the difference in the proportion of those reporting symptom relief in the study's treated group versus the placebo group wasn't that dramatic.
"It's statistically significant, yes. But is it clinically significant?"
While the findings shouldn't be dismissed, Cohen said he is cautious about the study because it was designed and funded by rifaximin's maker, Salix Pharmaceuticals Inc., a fact disclosed by the researchers. Pimentel [director of the Gastrointestinal Motility Program at Cedars-Sinai Medical Center] discovered the use of rifaximin for IBS, and Cedars-Sinai holds patent rights to this discovery and has licensed those rights to Salix.
Submitted by Paul Alper