Chance News 66

Quotations

"It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so."
--Mark Twain

Quoted by Richard H. Thaler in The overconfidence problem in forecasting, New York Times, 21 August 2010.

Submitted by Paul Alper

Forsooth

The following Forsooth is from the August 23, 2010 RRS News.

An editorial was published in the Journal of the National Cancer Institute (Volume 101, no 23, 2 December 2009). It announced some online resources for journalists, including a statistics glossary, which gave the following definitions:

• P value.  Probability that an observed effect size is due to chance alone.

  if p ≥ 0.05, we say 'due to chance', 'not statistically significant'

  if p < 05, we say 'not due to chance', 'statistically significant'

• Confidence interval (95% CI)

  Because the observed value is only an estimate of the truth, we know it has a 'margin of error'.

  The range of plausible values around the observed value that will contain the truth 95% of the time.

The journal subsequently (vol. 102, no. 11, 2 June 2010) published a letter commenting on the editorial and the statistics glossary. The authors of the original editorial replied as follows:

Dr Lash correctly points out that the descriptions of p values and 95% confidence intervals do not meet the formal frequentist statistical definitions.

[…] We were not convinced that working journalists would find these definitions user-friendly, so we sacrificed precision for utility.

Submitted by Laurie Snell

Risk reduction

Burger and a statin to go? Or hold that, please?
by Kate Kelland and Genevra Pittman, Reuters, 13 August 2010

Dr. Darrel Francis is the leader of "a study published in the American Journal of Cardiology, [in which] scientists from the National Heart and Lung Institute at Imperial College London calculated that the reduction in heart disease risk offered by a statin could offset the increase in risk from eating a cheeseburger and a milkshake."

Further, "When people engage in risky behaviors like driving or smoking, they're encouraged to take measures that minimize their risk, like wearing a seatbelt or choosing cigarettes with filters. Taking a statin is a rational way of lowering some of the risks of eating a fatty meal."

Discussion

1. Obviously, the above comments and analogies are subject for debate. Defend and criticize the comparisons made.

2. Risk analysis is very much in the domain of statistics. How would you estimate the risk of driving, smoking, eating a cheeseburger or taking a statin? Read the article in the American Journal of Cardiology to see how Francis and his co-authors estimated risk.

3. Pascal’s Wager is famous in religion, philosophy and statistics; his wager is the ultimate in risk analysis.

Historically, Pascal's Wager was groundbreaking as it had charted new territory in probability theory, was one of the first attempts to make use of the concept of infinity, [and] marked the first formal use of decision theory.

The wager is renowned for discussing the risk of not believing in God, presumably the Christian concept of God.

[A] person should wager as though God exists, because living life accordingly has everything to gain, and nothing to lose.

Read the Wikipedia article and discuss the risk tables put forth.

Submitted by Paul Alper

Teaching with infographics

Teaching with infographics: Places to start
by Katherine Schulten, New York Times, The Learning Network blog, 23 August 2010

Each day of this week, the blog will present commentary on some aspect of infographics. Starting with discussion of what infographics are, the series proceeds through applications ranging from sciences and social sciences to the fine arts. Each category will feature examples that have appeared in the Times. The link above for the first day contains an index to the whole series. There is a wealth of material to browse here.

Submitted by Bill Peterson

Subverting the Data Safety Monitoring Board

Don't Mess with the DSMB Jeffrey M. Drazen and Alastair J.J. Wood. N Engl J Med 2010; 363:477-478, July 29, 2010.

The Data Safety Monitoring Board (DSMB) is supposed to be an independent group charged with interim review of data from a clinical trial to decide whether to stop a study early because of evidence that continuation of the trial would be unethical. Trials are commonly halted early because of sufficient evidence that a new drug is clearly superior/inferior to the comparison drug, or because of serious concerns about safety.

The independent function of a DSMB is vital.

Since the DSMB (data and safety monitoring board) is charged with ensuring that clinical equipoise is maintained as trial data are accrued, it is considered very bad, even self-destructive, behavior for people who are involved with the study to interact with DSMB members on trial-related issues. Traditionally, there has been a wall between investigators, sponsors, and the DSMB. This wall prevents preliminary findings from leaking out in ways that would prejudice the trial. For example, if it was known that the DSMB was examining a marginal increase in cardiovascular risk in a trial, then trial investigators might bias future recruitment by excluding patients at risk for such events.

In the real world, though, problems with the DSMB occur. In one case, a drug company bypassed the DSMB and conducted an in-house examination of data in a trial and quickly published the data to counter a recently published meta-analysis that suggested safety issues associated with that company's drug. Why is this a problem?

The DSMB should have been informed of our May 2007 article and checked the trial data to be sure that patients receiving rosiglitazone in the RECORD trial were not having adverse events at an unacceptable rate. If clinical equipoise was still in play, the trial should have been allowed to continue undisturbed (i.e., without publication of the RECORD interim analysis), without public comment from the DSMB, without communication with investigators, and without disturbing the integrity of the trial. On the other hand, if in the opinion of the DSMB equipoise no longer existed, then the trial should have been terminated — that is the way it is supposed to work. The DSMB protects the participants in a trial.

This editorial described a second trial where interim data that should have been blinded from everyone except the DSMB became publicly known. A detailed description of this trial can be found in an earlier NEJM article.

Another concern about revelation of data in a DSMB involves the potential for insider trading. There is a nice description of the problems that disclosure can have in this Seattle Times article from 2005.

Questions

1. Some DSMBs analyze data that is blinded by coding the two arms of the study with generic letters like A and B. With generic letters, though, it still may be possible to guess which group is which. How?

2. There are also examples where the DSMB is presented only with aggregate data across both arms of the study. What types of safety issues could be analyzed with only aggregate data? What types of safety issues would be impossible to conduct with only aggregate data?

3. If the rules for stopping a study are specified in detail prior to data collection, would a DSMB still be needed?

Submitted by Steve Simon

Think the answer clear? Look again

The New York Times Science
by Katie Hafner
August 30, 2010

Here we read:

(1) Win an Academy Award and you’re likely to live longer than had you been a runner-up.

(2) Interview for medical school on a rainy day, and your chances of being selected could fall..

Win an Academy Award and you’re likely to live longer than had you been a runner-up.

Interview for medical school on a rainy day, and your chances of being selected could fall.

Such are some of the surprising findings of Dr. Donald A. Redelmeier, a physician-researcher and perhaps the leading debunker of preconceived notions in the medical world.

Dr. Redelmeier’s work on longevity began 10 years ago, when he was watching the Academy Awards and noticed that the celebrities on stage “don’t look anything like the patients I see in clinic,” he said. “It’s not just the makeup and the plastic surgery and wardrobe. It’s the way they move, it’s their gestures. They seem so much more vivacious. It seemed so much more than skin deep and might go all the way to longevity.”

His findings: Academy Award winners live an average of three years longer than the runners-up. A potential explanation could be an added measure of scrutiny, a public expectation of healthier living.

This was convincidlty debunked by Peter Doyle and his student Mark Mixer. Mark illustrated the key of their solution with the remark "breaking your hip increases your life" and he and Peter used a simulation to show that Oscer winners do not live longer.

You can see this in Winners do not live longer

Later this was also explained in another article to the Annals Of Internam Medicine

Do Oscar Winners Live Longer than Less Successful Peers? A Reanalysis of the Evidence Marie-Pierre Sylvestre, MSc; Ella Huszti, MSc; and James A. Hanley, PhD

Here we read "in an article published in Annals of Internal Medicine in 2001, Redelmeier and Singh reported that Academy Award–winning actors and actresses lived almost 4 years longer than their less successful peers. However, the statistical method used to derive this statistically significant difference gave winners an unfair advantage because it credited an Oscar winner's years of life before winning toward survival subsequent to winning. When the authors of the current article reanalyzed the data using methods that avoided this “immortal time” bias, the survival advantage was closer to 1 year and was not statistically significant.

As a condition of publication, we required the authors to make the data set available to interested researchers. Unfortunately, various complications prevented its prompt dissemination, and it has taken almost 5 years for someone to come forward with a reanalysis of the data. We are glad to publish Sylvestre and colleagues' reanalysis, partly because the article affords a chance to amend a widely publicized result, but more so because the analytic methods at issue apply to many health care research questions.

Is the United States a religious outlier?

Religious outlier by Charles Blow, The New York Times, September 4, 2010.

The following image was published on the New York Times website.

http://graphics8.nytimes.com/images/2010/09/04/opinion/0904OPEDBLOW_600sub.jpg

The author, Charles Blow, is the visual OpEd columnist for the New York Times. His comments about the graph are rather brief.

With all of the consternation about religion in this country, it’s sometimes easy to lose sight of just how anomalous our religiosity is in the world. A Gallup report issued on Tuesday underscored just how out of line we are. Gallup surveyed people in more than 100 countries in 2009 and found that religiosity was highly correlated to poverty. Richer countries in general are less religious. But that doesn’t hold true for the United States.

Questions

1. Does the United States look like an outlier to you? Are there any other outliers on this graph?

2. Why would there be a relationship between GDP and percentage of people who call themselves religious? Does a higher GDP cause lower religiosity? Does a lower religiosity cause a higher GDP? What sort of data could you collect that might help answer this question?

3. Do you like how Mr. Blow presented this data? What would you change, if anything, in this graph?

Submitted by Steve Simon

Additional Discussion

Notice how many dimensions are included in addition to the axes (percentage who say religion is important and G.D.P. per capita). The Gallup poll from which the graph came can be found here. Gallup says:

Results are based on telephone and face-to-face interviews conducted in 2009 with approximately 1,000 adults in each country. For results based on the total sample of national adults, one can say with 95% confidence that the maximum margin of sampling error ranges from ±5.3 percentage points in Lithuania to ±2.6 percentage points in India. In addition to sampling error, question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of public opinion polls.

What might be "practical difficulties in conducting surveys"? "wording difficulties"? What effect might these have on the findings?

Submitted by Paul Alper