Chance News 67

More fuel to feed the fiery controversy over mammograms

Mammogram Benefit Seen for Women in Their 40s, Gina Kolata, The New York Times, September 29, 2010.

One of the most contentious debates in medicine is whether mammograms are beneficial to women between 40 and 50 years old. Earlier commentaries about this controversy appear in Chance News 8, Chance News 12, Chance News 14, Chance News 47, Chance News 58, and Chance News 59.

The first sentence in the latest article about mammography makes a bold claim...

Researchers reported Wednesday that mammograms can cut the breast cancer death rate by 26 percent for women in their 40s.

...and the second sentence contradicts this claim.

But their results were greeted with skepticism by some experts who say they may have overestimated the benefit.

The data set on which these bold claims were based is quite good.

The new study took advantage of circumstances in Sweden, where since 1986 some counties have offered mammograms to women in their 40s and others have not, according to the lead author, Hakan Jonsson, professor of cancer epidemiology at Umea University in Sweden. The researchers compared breast cancer deaths in women who had a breast cancer diagnosis in counties that had screening with deaths in counties that did not. The rate was 26 percent lower in counties with screening.

Why the skepticism?

One problem, said Dr. Peter C. Gotzsche of the Nordic Cochrane Center in Copenhagen, a nonprofit group that reviews health care research, is that the investigators counted the number of women who received a diagnosis of breast cancer and also died of it. They did not compare the broader breast cancer death rates in the counties.

A prominent statistician, Donald Berry, is also quoted in this article.

Questions

1. The research design in the current study was not randomized. Is this an issue?

2. What are the barriers to conducting a randomized trial for mammography?

Proofiness

Charles Seife is a marvelous writer of serious, interesting topics for the lay reader:

• Zero: The Biography of a Dangerous Idea, 2000
• Alpha & Omega: The Search for the Beginning and End of the Universe, 2004
• Decoding The Universe, 2007
• Sun in a Bottle: The Strange History of Fusion and the Science of Wishful Thinking, 2008

His latest book, Proofiness: The Dark Arts of Mathematical Deception, 2010, makes for especially good reading for students and teachers of statistics. The following web sites all comment on the book: The New York Times has a review and an exerpt; NPR ran a story, Lies, Damned Lies, And 'Proofiness'; additional reviews appeared in New York Journal of Books and Politics Daily.

The reviews are entirely favorable, but don’t quite do justice to his presentation, so readers of Chance News are encouraged to read the book as well as the above commentaries.

Seife defines proofiness as “the art of using bogus mathematical arguments to prove something that you know in your heart is true — even when it’s not.” However, he never makes the connection to Innumeracy

A term meant to convey a person's inability to make sense of the numbers that run their lives. Innumeracy was coined by cognitive scientist Douglas R Hofstadter in one of his Metamagical Thema columns for Scientific American in the early nineteen eighties. Later that decade mathematician John Allen Paulos published the book Innumeracy. In it he includes the notion of chance as well to that of numbers.

Seife also does not refer to Stephen Colbert’s even more famous neologism, truthiness which

is a "truth" that a person claims to know intuitively "from the gut" without regard to evidence, logic, intellectual examination, or facts.

Colbert himself put truthiness this way: "We're not talking about truth, we're talking about something that seems like truth – the truth we want to exist."

Seife begins his Introduction with the famous quotation of Senator Joseph McCarthy on February 9, 1950:

"I have here in my hand a list of 205--a list of names that were made known to Secretary of State as being members of the Communist party and who nevertheless are still working and shaping policy in the State Department.

The 205 later became 57 and then 81. “It really didn’t matter whether the list had 205 or 57 or 81 names. The very fact that McCarthy had attached a number to his accusations imbued them with an aura of truth.” This “outrageous falsehood was given the appearance of absolute fact.”

Seife attempts to categorize the types of proofiness:

A. Potemkin numbers--numerical facades that look like real numbers such as crowd estimates or the number of communists in the State Department.
B. Disestimation, another neologism--“the act of taking a number too literally, understating or ignoring the uncertainties that surround it.”
C. Fruit packing--“it’s not the individual numbers that are false; it is the presentation of the data that creates the proofiness.”
D. Cherry picking--a form of fruit packing in which there is a “careful selection of data, choosing those that support the argument you wish to make while underplaying or ignoring data that undermine it.”
E. Apples to oranges comparison--another form of fruit packing, for example, comparing dollar amounts without taking into account inflation.
F. Apple polishing--another form of fruit packing, for example, deceptive graphs where the origin is missing; or, algebraically, misuse of mean and median.
G. Causuistry, another neologism and a pun on the word casuistry--“a specialized form of casuistry where the fault in the argument comes from implying that there is a causal relationship between two things when in fact there isn’t any such linkage.”
H. Randumbness, another neologism--“insisting that there is order where there is only chaos” or, “creating a pattern where there is none to see.”
I. Regression to the moon--for example, extrapolating instead of interpolating regression results.

None of these categories are new to teachers of statistics but his examples of the above forms of proofiness are detailed and when not frightening, and amusing; these examples include: the O.J. Simpson trial; the Franken-Coleman Minnesota Senate election and Bush vs. Gore in 2000 (he terms them “electile dysfunctions”); nuclear testing; risk analysis; the space program; the Vietnam war; and, determination of the perfect butt (page 66 contains the formula for callipygianness--a word which is not a neologism). He is particularly incisive when he contrasts systematic error when it overwhelms and confuses the notion of error due to sampling, and thus, invalidating the so-called margin of error in polling.

Discussion

1. If it is so obvious today that McCarthy was fabricating the numbers--in the parlance of today, he was fact-free--why was he so successful so long in the 1950s? And why did his allegations and point of view live on well after his death in 1957?

2. Seife devotes a great deal of time to convince the reader that the U.S. census would be more accurate if it did not attempt to count everyone but rather did statistical sampling and avoid many of the systematic errors. Why would this be true? Why did the U.S. Supreme Court deem otherwise?

3. Some of his strongest criticism is directed at journalists and polling organizations. The chapter entitled, “Poll Cats.” On page 109 he says, “Internet polls have no basis in reality whatsoever.” Why? “Yet, CNN.com has an Internet poll on its front page every day.” Again, why? Non Internet polls do not come off much better due to flagrant non-statistical faults.

4. With regard to the O.J. Simpson murder trial, Seife paraphrases one of Simpson's defense attorneys claim that “only one in a thousand wife-beaters winds up murdering his spouse. One in a thousand! Such a small probability means that O.J. Simpson almost certainly isn’t the murderer, right? “ Use Bayes theorem along with reasonable numbers about the number of wives being murdered to indicate that Simpson’s probability of being the culprit is much higher.

5. Regression to the moon also refers to totally nonsensical use of regression. A more detailed look (page 66) at callipygianness reveals

Callipygianness = (S + C) x (B + F) / (T - V) ,

where S is shape, C is circularity, B is bounciness, F is firmness, T is texture, and V is waist-to-hip ratio. Seife found this regression result, not surprisingly, on Fox News and the reporter was from another Murdoch enterprise, The New York Post. Why does Seife find this regression result so ridiculous? On the same page, there is a regression result for “Misery” which depends upon weather, debt, motivation, “the need to take action,” and some other variables. “[I]t proved --scientifically--that the most miserable day of the year [2005] was January 24.” The regression result for “Happiness” appears on the preceding page. Why does Seife claim that these three are examples of Potemkin numbers?

6. To return to McCarthy’s proofiness, his original speech about the 205 communists in the State Department was made in Wheeling, West Virginia to the Republican Women’s Club and made no waves whatsoever for days. Seife does not mention this, but only after the New York Times and the Washington Post publicized the speech did it ignite his fame. Contrast that time lag with today’s instant communication.

7. Seife on page 226 repeats a famous adage of the journalism world: “If your mother says she loves you, check it out.” He then looks at the Pentagon’s weekly body counts and monthly hamlet evaluations during the Vietnam War. By page 228 he describes an auto-industry market research report which shows that driving a Hummer H3 is “better for the environment than driving the energy-efficient Toyota Prius hybrid.” Why did he juxtapose these two examples?

8. The last paragraph of the book is: “Mathematical sophistication is the only antidote to proofiness and our degree of knowledge will determine whether we succumb to proofiness or fight against it. It’s more than mere rhetoric; our democracy may well rise or fall by the numbers.” Why might his “antidote” be insufficient?

Submitted by Paul Alper