Sandbox
Do Oscar Winners Live Longer?
If you put "Oscar winners live longer" in Google you will get over 7,000 hits. Here is one from the January 23, 2007 issue of Health and Aging
Oscar winners live longer: Reported by Susan Aldridge, PhD, medical journalist.
It is Oscar season again and, if you're a film fan, you'll be following proceedings with interest. But did you know there is a health benefit to winning an Oscar? Doctors at Harvard Medical School say that a study of actors and actresses shows that winners live, on average, for four years more than losers. And winning directors live longer than non-winners. Source: "Harvard Health Letter" March 2006.
The assertion that Oscar winners live longer was based on an article by Donald Redelmeier, and Sheldon Singh: "Survival in Academy Award-winning actors and actresses". Annals of Internal medicine, 15 May, 2001, Vol. 134, No. 10, 955-962.
This is the kind of study the media loves to report and medical journals enjoy the publicity they get. Another such claim, in the news as this is written, is that the outcome of the Super Bowl football game determines whether the stock market will go up or down this year. Unlike the Oscar winners story the author of this claim, Leonard Koppet, admited that it was all a joke, see Chance News 13.04.
A recent paper by James Hanley, Marie-Pierre Sylvestre and Ella Huszti, "Do Oscar winners live longer than less successful peers? A reanalysis of the evidence," Annals of Internal medicine, 5 September 2006, Vol 145, No. 5, 361-363, claims that the Redelmeier, Singh paper was flawed. They provided a reanalysis of the data showing that it does not support the claim that Oscar winners live longer.
For their study, Redelmeier and Singh identified all actors and actresses ever nominated for an academy award in a leading or a supporting role up to the time of the study (n = 762). Among these there were 235 Oscar winners. For each nominee another cast member of the same sex who was in the same film and was born in the same era was identified (n= 887) and these were used as controls.
The authors used the Kaplan-Meier method to provide a life table for the Oscar winners and the control group. A life table estimates for each x the probability of living x years. In Chance News 10.06 We illustrated the Kaplan-Meier method, using data obtained from Dr. Radelmeier.
In their paper Redelmeier and Singh provided the following graph showing the life tables of the two goups.
The areas under the two curves are estimates for the life expectance for the two groups. Using a test called the "log-rank test" they concluded that the overall difference in life expectancy was 3.9 years (79.7 vs. 75.8 years; P) = .003.
While the life tables look like standard life tables there is one big difference. We note that 100 percent of the Oscar winners live to be at least 30 years old. Of course this is not surprising because they are known to be Oscar winners. Thus we know ahead of time that the Oscar winners will live longer than a traditional life table would predict. This gives them an advantage in their life expectancy. This is called a selection bias. Of course the controls also have an advantage because we know that were in a movie at about the same age as a nominee. But there is no reason to believe that these advantages are the same.
Here is a more obvious example of selection bias discussed in Robert Abelson's book "Statistics as Principled Argument' and reported in Chance News 4.05.
A study found that the average life expectancy
of famous orchestral conductors was 73.4 years, significantly higher than the life expectancy for males, 68.5, at the time of the study. Jane Brody in her "New York Times" health column reported that this was thought to be due to arm exercise. J. D Caroll gave an alternative suggestion, remarking that it was reasonable to assume that a famous orchestra conductor was at least 32 years old. The life expectancy for a 32 year old male was 72 years making the 73.4
average not at all surprising.
To avoid the possible of selective bias, Redelmeier and Singh did an analysis using time-dependent covariates, in which winners were counted as controls until the time of first they won the Oscar. This resulted in a difference in life expentance of 20% (CI, 0% to 35%). Since the confidence interval includes 0 the difference is not significant. So one might wonder why they made their claim that Oscar winners live longer.
In a letter to the editor in response to the study by Hanley et al., Redelmeier and Singh report that they did the same analysis with one more year's data and obtained a result even more clearly not significant.
To avoid selection bias, Sylvester and colleagues analyzed the data by comparing the life expectancy of the winners from the moment they win with others alive at that age. In the McGill Press Release, Hanley remarks "The results are not as, shall we say, dramatic, but they're more accurate." We recommend reading this press release for more information about the study by Sylvester et al.
When the Redelmeier and Singh paper came out, our colleague Peter Doyle was skeptical of the results (he called their paper "a crock"). In the fall of 2003 he wrote to the editor of the Annals of Internal Medicine, asking if the article was a hoax:
It has finally dawned on me that, like the screenwriters study in the
British Medical Journal, the original OSCAR study (Annals of Internal Medicine, 15 May 2001) may not have been a travesty of statistics after all, but simply a joke. Here's why I think so.
(1) The conclusion---which has been colorfully paraphrased as saying that winning an OSCAR is better for your health then being made immune to cancer---is inherently preposterous. The suggestion seems really `too stupid to discuss'. This should alert the reader that the article may be a hoax.
(2) The fallacy upon which the conclusion is based is well-known, and instantly obvious in the case of similar claims, like Mark Mixer's keen observation that `breaking your hip increases your life expectancy'. We're not talking here about something really subtle---we're talking about a common statistical fallacy. The authors could reasonably expect that the fallacy would be evident (if not instantly then upon a little reflection) to anyone with basic statistical savvy---or just plain common sense.
(3) The article itself implicitly recognizes the fallacy behind the method; describes how to modify the method to avoid the fallacy (`time-dependent covariates'); and indicates that when the method is modified appropriately, the results lapse into statistical insignificance. However, all this is done subtly, so that it doesn't leap out at you. Unless you are on the lookout, you're likely either to miss the point entirely, or fail to appreciate its significance. The effect here is much the same as if the authors had dated the article `1 April 2001': Once you identify the fallacy that is at work here, you will find within the article itself the evidence to confirm that this is indeed a hoax.
(4) The article claims that data will be made available online, but when you look for the data, you find only a lame excuse for why it isn't available (something about `confidentiality'). The statement that data will be made available is something you would normally include in an article like this: It's all part of the masquerade. The authors didn't bother to make the data available because they didn't think anyone would be dense enough to actually want to look it.
In light of all this, it seems plausible that this article was intended to be a joke---at least by the authors.
The editor assured Peter that the article was not a hoax, and that it had been submitted to a "painstaking review".
To demonstrate the fallacy behind the article, Peter wrote a computer simulation, using Oscar data compiled by Mark Mixer. The program generated fake Oscar winners by choosing at random from among the nominees. These fake winners exhibited the same apparent tendency to live longer that is exhibited by the genuine winners.
Despite the fact that, in their paper Redelmeier and Singh said the data they used would be available on their website, it never was. That is why Peter had to rely on Mark for the data in his simulation. Mark's data is available here embedded in Peter's Mathematica program. If you do not have Mathematica you can read this using the free MathReader. (We will also make Mark's data available in Excel format.) For the paper by Hanley and his colleagues, Redelmeier and Singh did make their data available, though it was not the original data since it included the results of one more year of Oscars winners. This data is available here.
Peter simulation can also be used to show that even a supposedly correct application of the Kaplan-Meier test fails to eliminate selection bias completely: Effects to which the corrected test assigns a p-value less than .05 happen more than 5 percent of the time. Let's try to see why. Peter describes the corrected form of the Kaplan-Meier test as follows:
We decide to compare those who have
won an Oscar (call them ‘winners’) with those who have merely been nominated (call them ‘also-rans’). Our ‘null hypothesis’ is that having won an Oscar doesn’t help your health. We create a contest by associating a point to the death of anyone who has ever been nominated for an Oscar. Points are bad: the winners get a point if the deceased was a star; the also-rans get a point if the deceased was an also-ran. Suppose that the deceased died in their dth day of life. Over the course of history, some number a of nominees will have made it to the kth. day of their lives, and been a winner on that day; some number b of nominees will have made it to the dth day of their lives, and been an also-ran on that day. If our hypothesis is correct, and having won an Oscar really doesn’t help your health, then the probability that the winners get this point should be a/(a+b). So now we’ve got a game of points. with known probability of winning for each
point.
The corrected Kaplan-Meier statistic is obtained by analyzing this game as if it had been determined beforehand exactly how many points would be played, together with the associated parameters a and b. But of course this was not the case. There are subtle interdependencies which cause the method to generate misleading p-values.
This defect of the Kaplan-Meier test is known to experts in survival analysis. It would be worrisome, were it not for the fact that the errors it introduces are tiny compared to those resulting from the failure to understand that, as Mark Mixer so aptly observed, "breaking your hip increases your life expectancy."
Homework
Write a computer simulation showing why "breaking your hip increases your life expectancy", based on the simplest probability model you can come up with. Analyze the results of your simulation using the corrected Kaplan-Meier test, and observe that deviations to which the corrected test ascribes p<.05 happen more than 5 percent of the time.