Chance News 45

Quotations

Science is not the arbiter of truth. All it can do is offer opinions about the answers to certain questions that we ask of nature. And it reserves the right to revise those opinions in the light of future discoveries.

Even mathematics loses touch with any notion of truth once it steps into the real world. Last May, the director of the Max Planck Institute for Mathematics in Germany, warned that financial systems were operating in dangerous territory because traders were transferring their naive notions of the truth of mathematics on to the "black box" models used to predict and control trading. A few months later, we all found out just how dangerous that territory was.

Michael Brooks

The Guardian
Saturday, 24 January, 2009

Submitted by Laurie Snell

Forsooths

College Kids and Monkeys About Equal on Math

Robert Preidt
MSN Health & Fitness headline

Submitted by Paul Alper

---

The following Forsooth is from the March 2009 RSS NEWS:

The other kind of variable is called a quantitative variable in which numbers are used to order or to represent increasing levels of that variable. The simplest example of a quantitative variable is a dichotomous variable such as sex or genre, where one category is seen as representing more of that quality than the other. For example, if females are coded as 1 and males as 2, then this variable may be seen as reflecting maleness in which the higher score indicates maleness. The next simplest example is a variable consisting of three categories such as social class, which may comprise the three categories of upper, middle and lower. Upper class may be coded as 1, middle as 2 and lower as 3, in which case lower values represent higher social statuses. These numbers may be treated as a ratio measure or scale. Someone who is coded as 1 is ranked twice as high as someone who is coded as 2 giving a ratio of 1 to 2.

Advanced Quantitative Data Analysis
Open University Press, 2003

Fixing a "failed" airplane?

The financial engineers are at it again.

Critics may complain that these math wizards started the trouble in the first place by designing securities that couldn't withstand the market's turbulence. But they also may have the expertise to help fix the problem.

"Airplanes fail, too," says Peter Cotton, founder of Julius Finance, a structured-finance firm in New York. "That doesn't mean you don't fix them."

The Wall Street Journal, February 23, 2009
"Math Wizards Working On Spells to 'Cure'"

Scott Patterson

Submitted by Margaret Cibes

Intelligent dice?

March 4 headline: "Charts Suggest Dow Is Due For a Bounce"

March 13 headline: "The Dangerous Game of Predicting When Stocks Have Bottomed Out: Good Luck, Because Market Doesn't Care About Your Call"

The Wall Street Journal, March 2009

Submitted by Margaret Cibes

Is an intelligent mutual fund manager worth the cost

The Index Funds Win Again Mark Hulbert, The New York Times, February 21, 2009.

We pay mutual fund managers a substantial fee to invest our money intelligently. Hedge fund managers ask for even larger fees. Do we get value in return for this money? An alternative, the index fund, simply tries to match the return of the overall market and does not try to pick stocks that are expected to perform better than average. You could, for example, buy one share of every stock represented in the Standard and Poor's 500. Such a fund would never do better than average, but it would cost a lot less to administer because you would not be paying for a team of researchers to comb through the news reports to try to identify individual stocks or broad market sectors that are expected to perform better than average.

Most index funds do not purchase all 500 stocks in the S&P 500 (or all 3,000 stocks in the Russell 3000) but rather select a representative sample.

The data seems to indicate that actively managed funds do not do much better than index funds, especially after expenses are accounted for. A new study by Mark Kritzman appears to confirm this belief.

Expenses associated with a mutual fund are surprisingly difficult to calculate.

"The bite taken out by taxes, for example, depends on the specific combination of positive years and losing ones, as well as the order in which they occur. That combination and order also affect the performance fees charged by hedge funds."

The average actively managed fund and the average hedge fund did outperform the index fund before expenses, but

"For both the actively managed fund and the hedge fund, those expenses more than ate up the large amounts — 3.5 and 9 percentage points a year, respectively — by which they beat the index fund before expenses."

Of course, no one expects to select an average fund. If you pick a very well managed fund, is it likely to pay off?

"Mr. Kritzman calculates that just to break even with the index fund, net of all expenses, the actively managed fund would have to outperform it by an average of 4.3 percentage points a year on a pre-expense basis. For the hedge fund, that margin would have to be 10 points a year. The chances of finding such funds are next to zero, said Russell Wermers, a finance professor at the University of Maryland. Consider the 452 domestic equity mutual funds in the Morningstar database that existed for the 20 years through January of this year. Morningstar reports that just 13 of those funds beat the Standard & Poor’s 500-stock index by at least four percentage points a year, on average, over that period. That’s less than 3 out of every 100 funds."

What makes it worse is that these 3% of the funds were only obvious in hindsight. Picking a fund that will perform well in the future is a very difficult task. Keep in mind the warning that appears in most investment literature "past performance is no guarantee of future results".

The first popular criticism of the expenses associated with actively managed funds was a book by Burton Malkiel, "A Random Walk Down Main Street." Dr. Malkiel argues for the efficient market hypothesis, which states that the current prices of a stock represents all that is currently known about a stock, and that any changes represent a random walk.

The Wikipedia article on index funds offers a historical perspective on index funds. John Bogle started the first index fund in 1975. It was widely derided at the time.

"At the time, it was heavily derided by competitors as being 'un-American' and the fund itself was seen as 'Bogle's folly'. Fidelity Investments Chairman Edward Johnson was quoted as saying that he '[couldn't] believe that the great mass of investors are going to be satisfied with receiving just average returns'."

The fund, now called the Vanguard 500 Index Fund, is the most popular mutual fund available to the general public.

Submitted by Steve Simon

Questions

1. How would you pick a representative sample from the Standard and Poor's 500 or any other stock index?

2. What statistical principle is behind the saying that past performance does not guarantee future results?

3. Is it possible for a "great mass of investors" to experience above average returns?

Autism Statistics Lesson

Autism is a devastating disease. Recent attempts to pinpoint a cause have been recently in the news in the United States: New York Times article by Donald G. McNeil, Jr. and a follow-on New York Times editorial. The focus of these articles is on law suits regarding the measles, mumps and rubella (MMR) vaccine “or its combination with thimerosal, a mercury-based preservative that was used in most childhood vaccines until 2001,” as a cause of autism. After “5000 pages of testimony from experts and 939 medical articles,” judges concluded the plaintiffs failed to prove their assertions. One judge “ruled that the evidence was ‘overwhelmingly contrary’ to their argument.”

Coincidentally, in England autism was also in the news in February, here and here. In this instance, the story begins back in 1998 The Lancet, February, 1998 and ignores thimerosal but introduces a problem additional to autism due to the MMR vaccine, Crohn’s disease (inflammatory bowel disease). The Lancet article had an extraordinary impact on the general public in England as the following graph indicates:

http://www.dartmouth.edu/~chance/forwiki/autism.gif

MMR inoculation rates fall off sharply after the Lancet article and start to rise in 2004 because of a (London) Sunday Times investigation which revealed serious deficiencies in the Lancet study. These deficiencies often fall under the rubric of “follow the money,” a concept not given enough attention when discussing what constitutes statistical literacy.

The phrase, “follow the money,” is often thought to have originated in the book, All the President’s Men. According to Frank Rich the book never uses that phrase. It is however, from the film of the same name. Obviously, the pharmaceutical industry has a vested financial interest in vaccines and Deirdre Imus is suspicious of any “big pharma” vaccine and any doctor who sides with it. The main author of the Lancet article, Andrew Wakefield, unbeknownst to the twelve other authors of the Lancet study, had been paid “about $780,000 plus expenses, for his role in backing the generic case against MMR.” Further, he had a patent on “a single vaccine against measles—a potential competitor to MMR” which he claimed would cure “both inflammatory bowel disease and autism.” As cited by Glenn Frankel ten of the twelve other authors in 2004 issued a “Retraction of an interpretation” because “no causal link was established between MMR vaccine and autism.” Wakefield has since moved to the U.S. and according to a supporter of Wakefield, “The United States, with its privatized health care system and entrepreneurial spirit is much more fertile ground than Britain for a medical pioneer like Wakefield.” According to Brian Deer official figures showed that 1,348 confirmed cases of measles in England and Wales were reported last year [2008], compared with 56 in 1998. Two children have died of the disease.”

Discussion

1. If the medical profession overwhelmingly believes the MMR vaccine to be safe, why are parents of autistic children actively seeking litigation? That is, what element of emotional guilt might there be?

2. Although there were 13 authors of the Lancet article, there were only twelve children in the study. A multiplicity of authors is a common phenomenon in medical journals. Why is this so?

3. The Lancet article claims that in eight of the twelve children, “the average exposure to first behavioral symptoms was 6.3 days (range 1-14)” after receiving the MMR vaccine. An earlier version of the paper, not unearthed until 2005, puts the average at 14 days with the maximum time as 56 days. Further, it was later revealed that there was “no trace of measles virus [or mumps and rubella viruses] in any of the children.” Subsequent investigation indicated that instead of Crohn’s disease, the children were suffering from a benign condition, severe constipation. Moreover, the children were not randomly referred by general practitioners but were recruited from a lawyer “who had been attempting to raise a speculative lawsuit.” In 2007, Wakefield abandoned a libel claim, and agreed to pay costs, “estimated at about £500,000.” Assuming all of this is factually correct, explain why some parents still view Wakefield as a hero.

Submitted by Paul Alper

BBC Six-Part Primer on Understanding Statistics in the News

On the News & Announcements page from the American Statistical Association we read:

Last year, the BBC ran a six-part primer by Michael Blastland on understanding statistics in the news. Blastland takes on the media’s handling of surveys/polls, counting, percentages, averages, causation and doubt. “Wouldn't it be good,” Blastland said, “to have the mental agility to separate the wheat from the chaff?” He then proceeds, in six weekly articles, to point out the obvious vs. the correct ways to interpret the data.

Follow the links below to the BBC web site to read Michael Blastland’s six-part primer on understanding statistics in the news.

Surveys:

Counting:

Percentages:

Averages:

Causation:

Doubt:

Michael Blastland and Andrew Dilnot have written a book, What Are the Odds a Handy, Quotable Statistic Is Lying? Better Than Even which was recently reviewed in the New York Times by Barry Gewen. The review begins with:

It’s hard to resist a book that tells you that most people have more than the average number of feet. Or that researchers have found that Republicans enjoy sex more than Democrats do. Michael Blastland and Andrew Dilnot delight in bringing such facts to our attention — and then in explaining them away.

Discussion

How do you think the authors "explained away" these two examples?

Submitted by Laurie Snell

Most basketball statistics are worthless

The No-Stats All-Star Michael Lewis, The New York Times, February 15, 2009.

One of the best NBA basketball stars may have some of the worst statistics. Shane Battier of the Houston Rockets specializes in defense.

Battier has routinely ­guarded the league’s most dangerous offensive players — LeBron James, Chris Paul, Paul Pierce — and has usually managed to render them, if not entirely ineffectual, then a lot less effectual than they normally are. He has done it so quietly that no one really notices what exactly he is up to.

Shane Battier does not do any of the things that are likely to gain him public recognition.

His conventional statistics are unremarkable: he doesn’t score many points, snag many rebounds, block many shots, steal many balls or dish out many assists. On top of that, it is easy to see what he can never do: what points he scores tend to come from jump shots taken immediately after receiving a pass. “That’s the telltale sign of someone who can’t ramp up his offense,” Morey says. “Because you can guard that shot with one player. And until you can’t guard someone with one player, you really haven’t created an offensive situation. Shane can’t create an offensive situation. He needs to be open.” For fun, Morey shows me video of a few rare instances of Battier scoring when he hasn’t ­exactly been open. Some large percentage of them came when he was being guarded by an inferior defender — whereupon Battier backed him down and tossed in a left jump-hook. “This is probably, to be honest with you, his only offensive move,” Morey says. “But look, see how he pump fakes.” Battier indeed pump faked, several times, before he shot over a defender. “He does that because he’s worried about his shot being blocked.” Battier’s weaknesses arise from physical limitations. Or, as Morey puts it, “He can’t dribble, he’s slow and hasn’t got much body control.”

But he does help in the statistic that counts the most, wins.

The Grizzlies went from 23-59 in Battier’s rookie year to 50-32 in his third year, when they made the NBA playoffs, as they did in each of his final three seasons with the team. Before the 2006-07 season, Battier was traded to the Houston Rockets, who had just finished 34-48. In his first season with the Rockets, they finished 52-30, and then, last year, went 55-27 — including one stretch of 22 wins in a row.

The person who recognized Shane Battier's influence is Daryl Morey.

In 2005, the Houston Rockets’ owner, Leslie Alexander, decided to hire new management for his losing team and went looking specifically for someone willing to rethink the game. “We now have all this data,” Alexander told me. “And we have computers that can analyze that data. And I wanted to use that data in a progressive way. When I hired Daryl, it was because I wanted somebody that was doing more than just looking at players in the normal way. I mean, I’m not even sure we’re playing the game the right way.”

The first step in using statistics in basketball is recognizing which statistics are helpful and which ones are deceptive.

[T]he big challenge on any basketball court is to measure the right things. The five players on any basketball team are far more than the sum of their parts; the Rockets devote a lot of energy to untangling subtle interactions among the team’s elements. To get at this they need something that basketball hasn’t historically supplied: meaningful statistics. For most of its history basketball has measured not so much what is important as what is easy to measure — points, rebounds, assists, steals, blocked shots — and these measurements have warped perceptions of the game. (“Someone created the box score,” Morey says, “and he should be shot.”) How many points a player scores, for example, is no true indication of how much he has helped his team. Another example: if you want to know a player’s value as a ­rebounder, you need to know not whether he got a rebound but the likelihood of the team getting the rebound when a missed shot enters that player’s zone.

There is a wealth of discussion about selfish statistics, statistics that a player amasses to his own benefit, but to the detriment of the team. Shane Battier is effective because he is a very selfless player -- he makes choices that help his team rather than choices that pad his own statistics.

Submitted by Steve Simon

A rare event

Roger Woodard, Statistician at North Carolina University, wrote to Isostat :

It is rare that we see an article in the news that is willing to give you its data. There is an article in "The Big Money" that has a very nice graphic relating the ratings by critics and box office take of big blockbuster movies. It presents scatterplots, discusses correlations and regression lines. It is an article that should hold general interest for many students.

But more importantly this article allows you to download the data the authors used. And more amazingly, this article's last lines are "E-mail us, too, if you have any insights, questions, or complaints about the data or methodology. We're eager for as many people to play with these data as possible."

You can get the article (and link to the data)

here

Submitted by Laurie Snell

Videos on the economic crisis

In Chance News 43 we discussed an article in the New York Times entitled "Risk Mismanagement". This article introduced Nassim Nicholas Taleb, author of Fooled by Randomness and the Black Swan and gave his explanation for the world's current financial crisis. The article did not mention the well known mathematician Benoit Mandelbrot whose work on fractals provides the mathematics for Taleb's theories. Mandelbrot is the author of the book The (Mis) Behavior of Markets (A Fractal View of Financial Turbulence). They are good friends and you can listen here to an interview they gave on the economic crisis. You can find other interesting interviews on the state of the economic crisis at the same place, including one by Noam Chomsky.

Serious Medical Fraud

In an earlier Chance News wiki can be found a detailed treatment of the scientific fraud perpetrated by Eric Poehlman of the University of Vermont. In that same issue of Chance News there is a discussion of Poehlman predecessors who, it is claimed, were even more egregious producers of fraudulent data. But now we have another contender for the title, Scott S. Reuben of Tufts University and Baystate Medical Center.

According to Anesthesiology News,

Scott S. Reuben, MD, of Baystate Medical Center in Springfield, Mass., a pioneer in the area of multimodal analgesia, is said to have fabricated his results in at least 21, and perhaps many more, articles dating back to 1996. The confirmed articles were published in Anesthesiology, Anesthesia and Analgesia, the Journal of Clinical Anesthesia and other titles, which have retracted the papers or will soon do so, according to people familiar with the scandal. The journals stressed that Dr. Reuben's co-authors on those papers have not been accused of wrongdoing.

In addition to allegedly falsifying data, Dr. Reuben seems to have committed publishing forgery. Evan Ekman, MD, an orthopedic surgeon in Columbia, S.C., said his name appeared as a co-author on at least two of the retracted papers, despite his having had no hand in the manuscripts. "My names were forgeries on the documents," Dr. Ekman told Anesthesiology News.

The reason Reuben’s fraud is so serious is because

1. Dr. Reuben has been an extremely active and visible figure in multimodal analgesia, particularly as an advocate for its use in minimally invasive orthopedic and spine procedures. His research has provided support for several mainstays of current anesthetic practice, such as the use of nonsteroidal anti-inflammatory drugs [NSAIDs] and neuropathic agents instead of opioids and preemptive analgesia. Dr. Reuben has also published and presented data suggesting that multimodal analgesia can significantly improve long-term outcomes for patients.

2. In a recent Scientific American article, which dubbed Scott "A Medical Madoff", we read: "We are talking about millions of patients worldwide, where postoperative pain management has been affected by the research findings of Dr. Reuben," says Steven Shafer, editor in chief of the journal Anesthesia & Analgesia, which published 10 of Reuben's fraudulent papers.

Paul White, another editor at the journal, estimates that Reuben's studies led to the sale of billions of dollars worth of the potentially dangerous drugs known as COX2 inhibitors, Pfizer's Celebrex (celecoxib) and Merck's Vioxx (rofecoxib), for applications whose therapeutic benefits are now in question. Reuben was a member of Pfizer's speaker's bureau and received five independent research grants from the company. The editors do not believe patients were significantly harmed by the short-term use of these COX2 inhibitors for pain management but they say it's possible the therapy may have prolonged recovery periods.

From the Wall Street Journal:

The [Baystate Medical Center] hospital has asked the medical journals to retract the 21 studies, some of which reported favorable results from the use of painkillers like Pfizer Inc.'s Bextra and Merc & Co.'s Vioxx -- both since withdrawn -- as well as Pfizer's Celebrex and Lyrica. Dr. Reuben's research work also claimed positive findings for Wyeth's antidepressant Effexor XR as a pain killer. And he wrote to the Food and Drug Administration, urging the agency not to restrict the use of many of the painkillers he studied, citing his own data on their safety and effectiveness.

Discussion

1. The Anesthesiology News article provided this intriguing statistical insight: "Interestingly, when you look at Scott's output over the last 15 years, he never had a negative study," said one colleague, who spoke on the condition of anonymity. "In fact, they were all very robust results--where others had failed to show much difference. I just don't understand why anyone would do this or how anyone could pull this off for so long." How is this similar to the Madoff scandal?

2. The fraud was uncovered in a strange way. During a routine audit at Baystate, two of Reuben’s abstracts had not been approved by the hospital’s institutional review board (IRB), causing a possible breach of ethics because, whenever patients are involved, IRB approval is required. It turned out that IRB approval was not needed “because the data were fabricated” according to Dr. Jenson, chief academic officer of Baystate. “He told Anesthesiology News that simply put, Dr. Reuben had concocted the data—and in many cases the patients themselves—out of vapor.” Use Google to see the similarity and differences between this and that of the famous Cyril Burt fraud involving identical twins putatively separated at birth.

3. From Scientific American: In hindsight, Anesthesia & Analgesia editors Shafer and White admit that it should have been a "red flag" that Reuben's studies were consistently favorable to the drugs he studied. White, who has also received drug company educational grants, says that such funding comes with "subtle pressure" to give the companies the results they want. For now, at least, neither the drug companies nor Reuben's co-authors are officially sharing in the blame, but that's expected to change. "There's a lot of responsibility to pass around," White says, "It's all being focused on Scott Reuben, but the reality is there are many other responsible parties." What might be the subtle and not-so-subtle pressures of the sponsoring agencies and the institution itself?

4. Reuben’s journal articles are full of very small p-values and clinically significant effect sizes indicating that NSAIDs are safe and effective for pain medication after surgery. Indeed, his journal articles have the exquisite appearance of how proper comparisons should be presented. Below is a figure taken from an article supposedly co-authored with Ekman. What does this indicate about the peer review process?

http://www.dartmouth.edu/~chance/forwiki/cn451.gif This article has been retracted

Fig. 1 Illustration depicting the dose of morphine at each postoperative time-interval. The placebo group is represented by the straight line, and the celecoxib group is represented by the dashed line. The boxes represent the twenty-fifth to seventy-fifth percentiles, the horizontal lines represent the means, and the extended I-bars represent the fifth to ninety-fifth percentiles. Outliers, as shown by the solid circles, represent values that are >1.5 times the box length. The mean morphine dose at the first six postoperative time-intervals was significantly increased in the placebo group compared with the celecoxib group, as indicated by the double asterisks (p < 0.0001). The mean morphine dose at twenty-four hours was significantly increased in the placebo group compared with the celecoxib group, as indicated by the single asterisk (p < 0.003).

Submitted by Paul Alper

Losing at the half, winning in the end

The popular New York Times column Freakonomics had an interesting piece on basketball recently. Even more interesting is the academic article cited there: When Losing Leads to Winning, by Jonah Berger and Devin Pope of the University of Pennsylvania. The following graph illustrates their main point:

http://www.dartmouth.edu/~chance/forwiki/cn45.gif

That is, based on about 6500 NCAA basketball games, although being behind at the half is usually more likely to produce a loss, a one-point deficit at the half has a (surprisingly) higher probability of a win (51.3%) than being tied at the half (50%). They postulate that losing can lead to winning and cite a concept of “loss aversion.”

Discussion

1. Why is the above graph symmetrical about zero (game tied at the half)?

2. The following graph is not in the article itself but is courtesy of the authors:

http://www.dartmouth.edu/~chance/forwiki/CN45-2.gif

This graph, unlike the previous one, is not symmetrical about zero. However, regarding “When Losing Leads to Winning,” how is this graph similar to the previous one regarding the concept of “loss aversion”? What new ingredient makes it different?

3. The following punch line is from the New York Times: “It’s an intriguing finding: being behind by a little yields the greatest possible effort. And while these researchers measure these effects on the basketball court, or on pounding keyboards [other investigations done by Berger and Pope in their article], their implications for the rest of our lives are even more intriguing. Want your workers to work harder? Tell them that they are running a close second in the race for promotion.” Google Niccolò Machiavelli and see what he wrote on the subject of inspiring underlings to perform better.

Submitted by Paul Alper