Chance News 66

Quotations

"It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so."
--Mark Twain

Quoted by Richard H. Thaler in The overconfidence problem in forecasting, New York Times, 21 August 2010.

Submitted by Paul Alper

Forsooth

The following Forsooth is from the August 23, 2010 RRS News.

An editorial was published in the Journal of the National Cancer Institute (Volume 101, no 23, 2 December 2009). It announced some online resources for journalists, including a statistics glossary, which gave the following definitions:

• P value.  Probability that an observed effect size is due to chance alone.

  if p ≥ 0.05, we say 'due to chance', 'not statistically significant'

  if p < 05, we say 'not due to chance', 'statistically significant'

• Confidence interval (95% CI)

  Because the observed value is only an estimate of the truth, we know it has a 'margin of error'.

  The range of plausible values around the observed value that will contain the truth 95% of the time.

The journal subsequently (vol. 102, no. 11, 2 June 2010) published a letter commenting on the editorial and the statistics glossary. The authors of the original editorial replied as follows:

Dr Lash correctly points out that the descriptions of p values and 95% confidence intervals do not meet the formal frequentist statistical definitions.

[…] We were not convinced that working journalists would find these definitions user-friendly, so we sacrificed precision for utility.

Submitted by Laurie Snell

Risk reduction

Burger and a statin to go? Or hold that, please?
by Kate Kelland and Genevra Pittman, Reuters, 13 August 2010

Dr. Darrel Francis is the leader of "a study published in the American Journal of Cardiology, [in which] scientists from the National Heart and Lung Institute at Imperial College London calculated that the reduction in heart disease risk offered by a statin could offset the increase in risk from eating a cheeseburger and a milkshake."

Further, "When people engage in risky behaviors like driving or smoking, they're encouraged to take measures that minimize their risk, like wearing a seatbelt or choosing cigarettes with filters. Taking a statin is a rational way of lowering some of the risks of eating a fatty meal."

Discussion

1. Obviously, the above comments and analogies are subject for debate. Defend and criticize the comparisons made.

2. Risk analysis is very much in the domain of statistics. How would you estimate the risk of driving, smoking, eating a cheeseburger or taking a statin? Read the article in the American Journal of Cardiology to see how Francis and his co-authors estimated risk.

3. Pascal’s Wager is famous in religion, philosophy and statistics; his wager is the ultimate in risk analysis.

Historically, Pascal's Wager was groundbreaking as it had charted new territory in probability theory, was one of the first attempts to make use of the concept of infinity, [and] marked the first formal use of decision theory.

The wager is renowned for discussing the risk of not believing in God, presumably the Christian concept of God.

[A] person should wager as though God exists, because living life accordingly has everything to gain, and nothing to lose.

Read the Wikipedia article and discuss the risk tables put forth.

Submitted by Paul Alper

Teaching with infographics

Teaching with infographics: Places to start
by Katherine Schulten, New York Times, The Learning Network blog, 23 August 2010

Each day of this week, the blog will present commentary on some aspect of infographics. Starting with discussion of what infographics are, the series proceeds through applications ranging from sciences and social sciences to the fine arts. Each category will feature examples that have appeared in the Times. The link above for the first day contains an index to the whole series. There is a wealth of material to browse here.

Submitted by Bill Peterson

Subverting the Data Safety Monitoring Board

Don't Mess with the DSMB Jeffrey M. Drazen and Alastair J.J. Wood. N Engl J Med 2010; 363:477-478, July 29, 2010.

The Data Safety Monitoring Board (DSMB) is supposed to be an independent group charged with interim review of data from a clinical trial to decide whether to stop a study early because of evidence that continuation of the trial would be unethical. Trials are commonly halted early because of sufficient evidence that a new drug is clearly superior/inferior to the comparison drug, or because of serious concerns about safety.

The independent function of a DSMB is vital.

Since the DSMB (data and safety monitoring board) is charged with ensuring that clinical equipoise is maintained as trial data are accrued, it is considered very bad, even self-destructive, behavior for people who are involved with the study to interact with DSMB members on trial-related issues. Traditionally, there has been a wall between investigators, sponsors, and the DSMB. This wall prevents preliminary findings from leaking out in ways that would prejudice the trial. For example, if it was known that the DSMB was examining a marginal increase in cardiovascular risk in a trial, then trial investigators might bias future recruitment by excluding patients at risk for such events.

In the real world, though, problems with the DSMB occur. In one case, a drug company bypassed the DSMB and conducted an in-house examination of data in a trial and quickly published the data to counter a recently published meta-analysis that suggested safety issues associated with that company's drug. Why is this a problem?

The DSMB should have been informed of our May 2007 article and checked the trial data to be sure that patients receiving rosiglitazone in the RECORD trial were not having adverse events at an unacceptable rate. If clinical equipoise was still in play, the trial should have been allowed to continue undisturbed (i.e., without publication of the RECORD interim analysis), without public comment from the DSMB, without communication with investigators, and without disturbing the integrity of the trial. On the other hand, if in the opinion of the DSMB equipoise no longer existed, then the trial should have been terminated — that is the way it is supposed to work. The DSMB protects the participants in a trial.

This editorial described a second trial where interim data that should have been blinded from everyone except the DSMB became publicly known. A detailed description of this trial can be found in an earlier NEJM article.

Another concern about revelation of data in a DSMB involves the potential for insider trading. There is a nice description of the problems that disclosure can have in this Seattle Times article from 2005.

Questions

1. Some DSMBs analyze data that is blinded by coding the two arms of the study with generic letters like A and B. With generic letters, though, it still may be possible to guess which group is which. How?

2. There are also examples where the DSMB is presented only with aggregate data across both arms of the study. What types of safety issues could be analyzed with only aggregate data? What types of safety issues would be impossible to conduct with only aggregate data?

3. If the rules for stopping a study are specified in detail prior to data collection, would a DSMB still be needed?

Submitted by Steve Simon

Think the Answer Clear? Look Again

The New York Times

The article starts with

(1) Win an Academy Award and you’re likely to live longer than had you been a runner-up.

(2) Interview for medical school on a rainy day, and your chances of being selected could fall..

Such are some of the surprising findings of Dr. Donald A. Redelmeier, a physician-researcher and perhaps the leading debunker of preconceived notions in the medical world

It is hard to believe that the New York Times did not know that that item 1 itself has been debunked.

The assertion that Oscar winners live longer was based on [1]an article by Donald Redelmeier, and Sheldon Singh: "Survival in Academy Award-winning actors and actresses". of Internal medicine, 15 May, 2001, Vol. 134, No. 10, 955-962.

This was convincdenlty debunked here by Peter Doyle and his student Mark Mixer. Mark illustrated the key of their solution with the remark breaking your hip increases your life Peter used a simulation to show that Oscer winners do not live longer.

Later this was also explained by an article to the Annals Of Internam Meddicine

Do Oscar Winners Live Longer than Less Successful Peers? A Reanalysis of the Evidence Marie-Pierre Sylvestre, MSc; Ella Huszti, MSc; and James A. Hanley, PhD

n an article published in Annals of Internal Medicine in 2001, Redelmeier and Singh reported that Academy Award–winning actors and actresses lived almost 4 years longer than their less successful peers. However, the statistical method used to derive this statistically significant difference gave winners an unfair advantage because it credited an Oscar winner's years of life before winning toward survival subsequent to winning. When the authors of the current article reanalyzed the data using methods that avoided this “immortal time” bias, the survival advantage was closer to 1 year and was not statistically significant.

As a condition of publication, we required the authors to make the data set available to interested researchers. Unfortunately, various complications prevented its prompt dissemination, and it has taken almost 5 years for someone to come forward with a reanalysis of the data. We are glad to publish Sylvestre and colleagues' reanalysis, partly because the article affords a chance to amend a widely publicized result, but more so because the analytic methods at issue apply to many health care research questions. L