Chance News 17: Difference between revisions

From ChanceWiki
Jump to navigation Jump to search
Line 117: Line 117:
Norton Starr
Norton Starr


The College Mathematics Journal (CMJ Media) Highlights cover mathematics generally and their reviews often involve probability or statistical concepts so Chance News would enjoy these reviews.  As with this review, the probability and statistical contributions are often written by Norton Starr who has been a great help to Chance News.
The ''College Mathematics Journal'' (CMJ) Media Highlights covers mathematics generally and their reviews often involve probability or statistical concepts so Chance News readers would enjoy these reviews.  As with this review, the probability and statistical contributions are usually written by Norton Starr who has been a great help to Chance News.


Here Norton reviews a special section of [http://www.imstat.org/sts/issue_20_3.html ''Statistical Science'', August 2005], that recognized the 50th birthday of Daarrell Huff's famous book ''How to Lie with Statistics'' by asking several authors to contribute the following articles to this birthday party:
Here Norton reviews a special section of [http://www.imstat.org/sts/issue_20_3.html ''Statistical Science'', August 2005] that recognized the 50th birthday of Darrell Huff’s famous book "How to Lie with Statistics" by asking several authors to contribute the articles to this birthday party. These articles are:


Darrell Huff and Fifty Years of How to Lie with Statistics, Michael Steele.
"Darrell Huff and Fifty Years of How to Lie with Statistics", Michael Steele.
 
 
Lies, Calculations and Constructions: Beyond How to Lie with Statistics,  Joel Best.
"Lies, Calculations and Constructions: Beyond How to Lie with Statistics",  Joel Best.
 
 
Lying with Maps, Mark Monmonier.
"Lying with Maps", Mark Monmonier.
 
 
How to Confuse with Statistics or: The Use and Misuse of Conditional Probabilities, Walter Krämer and Gerd Gigerenzer.
"How to Confuse with Statistics or: The Use and Misuse of Conditional Probabilities", Walter Krämer and Gerd Gigerenzer.
 
 
How to Lie with Bad Data, Richard D. De Veaux and David J. Hand.
"How to Lie with Bad Data", Richard D. De Veaux and David J. Hand.
 
 
How to Accuse the Other Guy of Lying with Statistics, Charles Murray.
"How to Accuse the Other Guy of Lying with Statistics", Charles Murray.


Ephedra, Sally C. Morton.
"Ephedra", Sally C. Morton.


In Search of the Magic Lasso: The Truth About the Polygraph, Stephen, E. Fienberg and Paul C. Stern.
"In Search of the Magic Lasso: The Truth About the Polygraph, Stephen", E. Fienberg and Paul C. Stern.


Norton gives a brief discription of each of the papers.  We shall give only a few quotes from the articles that we found particularly interesting.  Michele Steeles tells us the story of the life of Darrel Huff and begins with:
Norton gives a brief description of each of the papers.  Rather than review his review we give quotes from three of the articles that we found particularly interesting.   
 
Michael Steeles tells us the story of the life of Darrel Huff and begins with:


<blockquote> In 1954 former ''Better Homes and Gardens'' editor
<blockquote> In 1954 former ''Better Homes and Gardens'' editor
and active freelance writer Darrell Huff published a
and active freelance writer Darrell Huff published a
slim (142 page) volume which over time would become
slim (142 page) volume, which over time would become
the most widely read statistics book in the history
the most widely read statistics book in the history
of the world. <br><br>
of the world. <br><br>
Line 150: Line 152:
for excellence in communication, and he had an exceptional
for excellence in communication, and he had an exceptional
commitment to doing things for himself.</blockquote>
commitment to doing things for himself.</blockquote>
After discussing the failure of the "critical thinking" movement in the late 1980's and the 1990's and asking "who would teach it” concludes his article with:
We all know statistical literacy is an important problem,
but we’re not going to be able to agree on its place
in the curriculum.<br><br>
Which means that How to Lie with Statistics is going
to continue to be needed in the years ahead.
When I read the book "The Bell Curve" by Richard Herrnstein and Charles Murray" to review for Chance News it seemed to me that the reviewers in the major newspapers could not have actually read the book.
In his article Charles Murray explains six methods to nock down a book that he describes as:
<blockquote> Tough but effective strategies for making people think that the target book is an irredeemable mess, the findings are meaningless, the author is incompetent and devious and the book’s thesis is something it isn’t.
In light of my experience with the Bell Curve I was particularly interested in Murray;s explanation of "Method 6".
"THE JUDICIOUS USE OF THE BIG LIE."
Finally, let us turn from strategies based on halftruths
and misdirection to a more ambitious approach:
to borrow from Goebbels, the Big Lie.
The necessary and sufficient condition for a successful
Big Lie is that the target book has at some point
discussed a politically sensitive issue involving gender,
race, class or the environment, and has treated this issue
as a scientifically legitimate subject of investigation
(note that the discussion need not be a long one, nor is
it required that the target book takes a strong position,
nor need the topic be relevant to the book’s main argument).
Once this condition is met, you can restate the
book’s position on this topic in a way that most people
will find repugnant (e.g., women are inferior to men,
blacks are inferior to whites, we don’t need to worry
about the environment), and then claim that this repugnant
position is what the book is about.<br><br>
What makes the Big Lie so powerful is the multiplier
effect you can get from the media. A television news
show or a syndicated columnist is unlikely to repeat
a technical criticism of the book, but a nicely framed
Big Lie can be newsworthy. And remember: It’s not
just the public who won’t read the target book. Hardly
anybody in the media will read it either. If you can get
your accusation into one important outlet, you can start
a chain reaction. Others will repeat your accusation,
soon it will become the conventional wisdom, and no
one will remember who started it. Done right, the Big
Lie can forever after define the target book in the public
mind.
Finally I agree with Norton's final remark in his review of the ''Statitistical Science'' articles:
The articles are both a compliment to and a complement of Huff's pathbreaking venture in writing.  This issue of ''Statistical Science'' is destined to be a collector's item.


==Put next time here==
==Put next time here==

Revision as of 16:12, 14 May 2006

Quotation

Put quotation here.

Put reference here.

Forsooths

Part of the fun of looking at Forsooths is trying to figure out why they are Forsooths. You should certainly try but if you get stumped you can read one person's idea of why they are Forsooths at the end of this Chance News.

The first three Forsooths are from the May 2006 RSS News.

Of the US Fortune 500 companies, 84 percent now have women on their boards: in the UK among the directors of companies in the FTSE 100, only 9 percent are women.


The Observer

19 March 2006


Thursday is the least productive day for finance workers, research has found, The start of the week is the best time with 18 per cent claiming they were most productive on a Monday.

Metro

26 January 2006


Question:

Kim has three vases in her living room, each containing the same number of flowers. Kim adds three fresh flowers to one vase which now has two more than the new average. How many flowers were in the vases orginally?

2006 Mensa puzzle calander

[note: answer given as "six", which is quite correct of course.]


Peter Winkler pointed out the the following question is not a forsooth:

Kim has *some* vases in her living room, each containing the same number of

flowers. Kim adds three fresh flowers to one vase which now has two more than

the new average. How many *vases* are there?

Walking on Water

For the most part, scientists, mathematician and statisticians labor in obscurity. Almost all of what they do is of no interest to the general public. The exception used to be if sex could somehow get connected and then the scientist/mathematician/statistician would suddenly be on the roladexes of the various talk-show programs. As an example, not so long ago a statistical study regarding the size of the ratio of the length of the forefinger to the ring finger was everywhere and anywhere. Why? Because the authors[Nature, 30 March, 2000] claimed there was a statistical significance for the difference of the ratio for homosexuals as compared to heterosexuals. Thus, an easy noninvasive, visual way of spotting sexual preference. The flaws in the study were numerous. The participants were chosen from gay pride celebrations in the vicinity of San Francisco, an area not known to be typical of the United States; multiple comparisons were made and with enough data dredging it is not statistically surprising that there would be the odd comparison that had a p-value less than 5% . The clinical (substantive, practical) significance was more or less zero in keeping with the negligible effect size coupled with measurement error. Nevertheless, titillation was high enough for several weeks of joking, hand comparisons and bad puns by the public and the media.

But sex, while always interesting, has given way to religion in American life. The phenomenal success of Dan Brown's The Da Vinci Code and the rise of the religious right guarantee that any scientific/mathematical/statistical research which can be tied to the Bible will bring instant celebrityhood. Even when the investigation appears in the unlikely Journal of Paleolimnology [2006 35:417-439] and involves "a small freshwater lake (148 km squared and a mean depth of 20 m)." The current name is Lake Kinneret but in Biblical days it was known as the Sea of Galilee upon which Jesus is said to have performed one of his miracles, walking on water. To walk on water is now a phrase that has come into the English language as being synonymous with extra-human, divine talent.

The paper by Nof, McKeague and Paldor is not an easy read, combining as it does analysis based on sea surface temperature, (warm and salty) springs, plume dynamics, ice dynamics and time series. The paper would never have made the talk-show circuit if it were only the typically dry--no pun intended-- presentation in such a technical journal. What sets it apart is its scientific explanation of how Jesus could manage to walk on water. In essence, after much physics, mathematics, and a bit of statistics, the authors have "proposed that the unusual local freezing process might have provided an origin to the story that Christ walked on water. Since the springs ice is relatively small, a person standing or walking on it may appear to an observer situated some distance away to be 'walking on water'." To avoid being inundated by hate mail (which they received in any event) they carefully state, "Whether this [walking on ice] happened or not is an issue for religion scholars, archeologists, anthropologists and believers to decide on."

In essence, the result of most of the highly mathematical argument in the paper is that things were occasionally colder back then and ice could have formed every once in a while, about every 160 years. Strangely enough, much of their data for this allegation comes from two core samples of temperature taken 2000 km away. The justification for this strange assertion is "because this distance is not any greater than the typical weather system scale in this part of the world." They do have some data much closer to the Lake but only from 1986 to 2003 yet "only the first 9 years of data were deemed suitable for use in the subsequent model." Because "the residual plots displayed some wild transitory behavior (as often seen for example, in financial time series data)," so they "added "a GARCH(1,1) component" to an AR(3) model resulting in the prediction of ice formation about every 160 years.

In their summary, the authors carefully state, "We hesitate to draw any conclusions regarding the implications of this study to the actual events that took place...Our springs ice calculations may or may not be related to the origin of the account of Christ walking on water." Nonetheless, Nof and Paldor are not strangers to conjuring up scientific explanations for Biblical phenomena. In 1992 they wrote an article, "Are There Oceanographic Explanations for the Israelites' Crossing of the Red Sea?" [Bulletin American Meteorological Society, 73; 305-314] This time, instead of temperature, it is wind which parted the Red Sea just long enough: "It is suggested that the crossing occurred while the water receded and that the drowning of the Egyptians was of a result of the rapidly returning wave." Nof likened this event to "It's like blowing across the top of a cup of coffee. The coffee blows from one end of the cup to the other." Statistics are completely absent in this paper. However, in 1993 they published a paper, "Statistics of Wind over the Red Sea with Application to the Exodus Question" [Journal of Applied Meteorology, 33, No 8; 1017-1025]. Here they "used the Weibull distribution ...applied to winds in the part of the Indian Ocean adjacent to the Red Sea" to argue that the likelihood of a proper storm would occur "roughly once every 2000 years."

---DISCUSSION---

1. Someone commented that "The reaction among Biblical scholars to Nof's theory ranged from bemused detachment to real irritation." Why the detachment and why the irritation?

2. Were the Israelites lucky to have picked the exactly correct moment? What calculations do you believe they did?

3. What physical phenomenon could explain the destruction of the walls of Jericho? Noah's flood? The Biblical burning bush?

4. The conflict between Darwinism and Biblical fundamentalism has been much in the news the past few years. Why hasn't there been any clash between fundamentalism and aspects of chemistry such as Avogadro's number?

Submitted by Paul Alper

Measuring poverty in London over 100 years

There goes the neighbourhood, From The Economist print edition, May 4th 2006.
Booth redux, From Economist.com, May 4th 2006.

This on-line article uses recent census data to graphically update a 100-year old map of poverty in London by district and street. The original project, led by the shipping magnate Charles Booth, colour-coded every street in the capital according to its social make-up. It shows the extent to which poverty depends on location and how little has changed over the past century.

The article illustrates one area, north Chelsea, in 1898 and 2001, colour-coding each street as either wealthy, well-off, middling or poor. In 1898, Chelsea was socially mixed, neither especially rich nor especially poor. Today Chelsea is considered a very desirable place to live, with many wealthy streets and some of the poverty has disappered. But on closer inspection the Economist claims that

poverty has not been altogether banished from this part of Chelsea, nor has it moved much. Most of the poorest areas in 2001 were also poor in 1898, and in almost exactly the same places. The reason is that the worst Victorian slums have been knocked down and replaced with tracts of social housing.

Neither the original survey nor its updated version use complicated statistical models. In 1898, researchers peered through windows and into back gardens, or asked police officers for opinions, in order to classify each street into one of seven categories from wealthy at the top to 'vicious, semi-criminal' at the bottom of the poverty scale. The 2001 census measures people's socio-economic status as one of eight categories. So to combine the two datasets a subset of four categories was used by the Economist. Having calculated the number of people, within the smallest unit available from the 2001 census, who fall into the four new categories, the single largest group is taken to represent the character of the area.

Questions

  • The Ecomonist gives an example of its classification methodology: if an output area contains 80 members of the upper managerial and professional class 'the wealthy' and 60, 40, and 20 members, respectively, of the other three new categories, it is taken to be wealthy. Is it reasonable to based the classification of an area on the most common category of resident? e.g. should the number of people in each steet be taken into account?
  • How might missing data be handled, old streets that have disappeared or new streets that didnt exist in 1898?

Further reading

  • The Charles Booth Online Archive is a searchable resource giving access to archive material from the Booth collections of the British Library of Political and Economic Science (the Library of the London School of Economics and Political Science) and the University of London Library.
  • Poverty maps of London - this interactive webpage allows viewers to zoom in on an area of London to see the original 1898 map juxtaposed with a modern view of the same area.
  • 2001 UK census

Submitted by John Gavin

Review of How to Lie with Statistics Turns Fifty.

Media Highlights, The College Mathematics Journal, Vol. 37, No 3, May 2006
Norton Starr

The College Mathematics Journal (CMJ) Media Highlights covers mathematics generally and their reviews often involve probability or statistical concepts so Chance News readers would enjoy these reviews. As with this review, the probability and statistical contributions are usually written by Norton Starr who has been a great help to Chance News.

Here Norton reviews a special section of Statistical Science, August 2005 that recognized the 50th birthday of Darrell Huff’s famous book "How to Lie with Statistics" by asking several authors to contribute the articles to this birthday party. These articles are:

"Darrell Huff and Fifty Years of How to Lie with Statistics", Michael Steele.

"Lies, Calculations and Constructions: Beyond How to Lie with Statistics", Joel Best.

"Lying with Maps", Mark Monmonier.

"How to Confuse with Statistics or: The Use and Misuse of Conditional Probabilities", Walter Krämer and Gerd Gigerenzer.

"How to Lie with Bad Data", Richard D. De Veaux and David J. Hand.

"How to Accuse the Other Guy of Lying with Statistics", Charles Murray.

"Ephedra", Sally C. Morton.

"In Search of the Magic Lasso: The Truth About the Polygraph, Stephen", E. Fienberg and Paul C. Stern.

Norton gives a brief description of each of the papers. Rather than review his review we give quotes from three of the articles that we found particularly interesting.

Michael Steeles tells us the story of the life of Darrel Huff and begins with:

In 1954 former Better Homes and Gardens editor

and active freelance writer Darrell Huff published a slim (142 page) volume, which over time would become the most widely read statistics book in the history of the world.

There is some irony to the world’s most famous statistics book having been written by a person with no formal training in statistics, but there is also some logic to how this came to be. Huff had a thorough training for excellence in communication, and he had an exceptional

commitment to doing things for himself.

After discussing the failure of the "critical thinking" movement in the late 1980's and the 1990's and asking "who would teach it” concludes his article with:

We all know statistical literacy is an important problem, but we’re not going to be able to agree on its place in the curriculum.

Which means that How to Lie with Statistics is going to continue to be needed in the years ahead.

When I read the book "The Bell Curve" by Richard Herrnstein and Charles Murray" to review for Chance News it seemed to me that the reviewers in the major newspapers could not have actually read the book.

In his article Charles Murray explains six methods to nock down a book that he describes as:

Tough but effective strategies for making people think that the target book is an irredeemable mess, the findings are meaningless, the author is incompetent and devious and the book’s thesis is something it isn’t.

In light of my experience with the Bell Curve I was particularly interested in Murray;s explanation of "Method 6".

"THE JUDICIOUS USE OF THE BIG LIE."

Finally, let us turn from strategies based on halftruths and misdirection to a more ambitious approach: to borrow from Goebbels, the Big Lie. The necessary and sufficient condition for a successful Big Lie is that the target book has at some point discussed a politically sensitive issue involving gender, race, class or the environment, and has treated this issue as a scientifically legitimate subject of investigation (note that the discussion need not be a long one, nor is it required that the target book takes a strong position, nor need the topic be relevant to the book’s main argument). Once this condition is met, you can restate the book’s position on this topic in a way that most people will find repugnant (e.g., women are inferior to men, blacks are inferior to whites, we don’t need to worry about the environment), and then claim that this repugnant position is what the book is about.

What makes the Big Lie so powerful is the multiplier effect you can get from the media. A television news show or a syndicated columnist is unlikely to repeat a technical criticism of the book, but a nicely framed Big Lie can be newsworthy. And remember: It’s not just the public who won’t read the target book. Hardly anybody in the media will read it either. If you can get your accusation into one important outlet, you can start a chain reaction. Others will repeat your accusation, soon it will become the conventional wisdom, and no one will remember who started it. Done right, the Big Lie can forever after define the target book in the public mind.

Finally I agree with Norton's final remark in his review of the Statitistical Science articles:

The articles are both a compliment to and a complement of Huff's pathbreaking venture in writing. This issue of Statistical Science is destined to be a collector's item.

Put next time here

Why the Forsooths are Forsooths.

(1) Letter to the editor: The Observer, March 26, 2006.

In the story 'Where women get real respect' (News, last week), you said: 'Of the US Fortune 500 companies, 84 per cent now have women on their boards; in the UK among directors of companies in the FTSE 100, only 9 per cent are women.' So what?

If every FTSE 100 company had 11 board members, and one of those was a woman, then 100 per cent of FTSE 100 companies would have a female board member and still only 9 per cent would be women.

If 84 per cent of F500 companies have a woman on the board, and every board has 20 members, then (about) 4 per cent of F500 board members are women.

Meaningless comparisons do not make an argument.
Jeremy Miles

University of York


(2) Zack Says:
March 10th, 2006
Digital Home of Zack Stewart >> Puzzled

n = the original number of flowers in each vase.

So after Kim adds 3 flowers to one vase it contains n+3 flowers.

The new average is thus (n+n+n+3)/3 = (3n+3)/3 = n+1 flowers.

So the special vase has (n+3) - (n+1) = 2 flowers more than the new average.

All of the above is true for any n.

I have to wonder what made them pick 6 as their answer - I would have gone for something interesting, like 5930912377. That way, when you turn the page over you at least get some fun shock value before you realize they’re full of it.


Peter Winkler's assumes that their are k vases each with n flowers. Kim adds 3 flowers to one of the vases.

Thus the special vase contains n+3 flowers making the new average (kn+3)/k = n + 3/k

Since the the special vase has 2 flowers more that the average n+3 = n + 3/k +2 so 1 = 3/k and so there are 3 vases.