Chance News 49: Difference between revisions
(36 intermediate revisions by 5 users not shown) | |||
Line 103: | Line 103: | ||
Fisher's exact test: P-Value = 0.032 | Fisher's exact test: P-Value = 0.032 | ||
However, these discrepancies are hardly in the Benford class. They may merely indicate what happens when a non-statistician medical doctor acts alone. | However, these discrepancies are hardly in the Benford class. They may merely indicate what happens when a non-statistician medical doctor acts alone.<br> | ||
Submitted by Paul Alper | Submitted by Paul Alper<br> | ||
For nice video about Benford's Law, see online lecture by Mark Nigrini, of the Cox School of Business [http://www.dartmouth.edu/~chance/ChanceLecture/NIGRIN/NIGRIN.HTM], from the Chance Lecture Series 2000.<br> | |||
Submitted by Margaret Cibes<br> | |||
==Emotional biases in financial decisions== | ==Emotional biases in financial decisions== | ||
Line 124: | Line 127: | ||
==Guesstimation== | ==Guesstimation== | ||
[http://www.nytimes.com/2009/03/31/science/31angi.html?scp=1&sq=biggest%20puzzles&st=cse The biggest of puzzles brought down to size]. <br> | [http://www.nytimes.com/2009/03/31/science/31angi.html?scp=1&sq=biggest%20puzzles&st=cse The biggest of puzzles brought down to size]. <br> | ||
New York Times, 30 March 2009 <br> | by Natalie Angier, ''New York Times'', 30 March 2009 <br> | ||
The article opens by reminding us that with bank bailouts running hundreds of billions of dollars the national debt passing ten trillion, the public need help comparing the magnitudes of really large numbers. For practice, the author recommends so-called "Fermi problems,". Named for Enrico Fermi, these are estimation problems that physicists and engineers like to use to sharpen their intuition. Two examples cited in the article are: | The article opens by reminding us that with bank bailouts running hundreds of billions of dollars the national debt passing ten trillion, the public need help comparing the magnitudes of really large numbers. For practice, the author recommends so-called "Fermi problems,". Named for Enrico Fermi, these are estimation problems that physicists and engineers like to use to sharpen their intuition. Two examples cited in the article are: | ||
Line 193: | Line 195: | ||
This article discusses concerns about lead content in lipsticks. Some doctors and others believe that lipsticks contain high levels of lead, while the FDA believes that any lead content would merely be a harmless trace. Doctors also disagree about whether there is any "safe" level of lead.<br> | This article discusses concerns about lead content in lipsticks. Some doctors and others believe that lipsticks contain high levels of lead, while the FDA believes that any lead content would merely be a harmless trace. Doctors also disagree about whether there is any "safe" level of lead.<br> | ||
In 2007 a study by a citizens' advocacy group found that "one-third of 33 lipsticks had lead in excess of 0.1 parts per million, the federal limit for candy." | In 2007 a study [http://www.safecosmetics.org/downloads/A%20Poison%20Kiss_report.pdf] by a citizens' advocacy group, Campaign for Safe Cosmetics, found that "one-third of 33 lipsticks had lead in excess of 0.1 parts per million, the federal limit for candy." | ||
<blockquote>Among the worst offenders were L’Oreal Colour Riche “True Red” lipstick (with a lead content of 0.65 parts per million) and Cover Girl’s Incredifull Lipcolor “Maximum Red” (0.56 p.p.m.). Price had nothing to do with lead levels: less expensive brands, like a $1.99 tube of Wet and Wild Mega Colors “Cherry Blossom,” contained no lead, whereas a $24 tube of Dior Addict “Positive Red” [since discontinued] contained 0.21 p.p.m.</blockquote> | <blockquote>Among the worst offenders were L’Oreal Colour Riche “True Red” lipstick (with a lead content of 0.65 parts per million) and Cover Girl’s Incredifull Lipcolor “Maximum Red” (0.56 p.p.m.). Price had nothing to do with lead levels: less expensive brands, like a $1.99 tube of Wet and Wild Mega Colors “Cherry Blossom,” contained no lead, whereas a $24 tube of Dior Addict “Positive Red” [since discontinued] contained 0.21 p.p.m.</blockquote> | ||
Manufacturers claim that their cosmetics are safe because they satisfy FDA requirements: manufacturers are only required to list "intended ingredients," not "unintended byproducts" of a manufacturing process, such as lead. Nevertheless, the advocacy group wants the FDA to release its data and to set a safety standard for lead in lipstick, not wait for a "peer-reviewed journal to publish its study of lead in lipstick."<br> | Manufacturers claim that their cosmetics are safe because they satisfy FDA requirements: manufacturers are only required to list "intended ingredients," not "unintended byproducts" of a manufacturing process, such as lead. Nevertheless, the advocacy group wants the FDA to release its data and to set a safety standard for lead in lipstick, not wait for a "peer-reviewed journal to publish its study of lead in lipstick."<br> | ||
The editor of [http://www.stats.org Stats], at George Mason University's Center for Health and Risk Communication, | The editor of [http://www.stats.org Stats], at George Mason University's Center for Health and Risk Communication, is quoted in the article: "These things sound terribly scary, but there’s a massive disconnect between how toxicologists evaluate risks and how activist groups evaluate risk, and even then there are debates.” In a March survey of over 900 members of the Society of Toxicology, 66% disagreed that cosmetics are a "significant source of chemical health risk," while 26% agreed and 8% "didn't know."<br> | ||
Submitted by Margaret Cibes<br> | Submitted by Margaret Cibes<br> | ||
==Swine flu pandemonium== | ==Swine flu pandemonium I== | ||
[http://www.courant.com/health/hc-swine-flu-pandemic-0612.artjun12,0,3556431.story "Connecticut Records Its Second Swine Flu Death"], by Arielle Levin Becker, The Hartford Courant, June 12, 2009<br> | [http://www.courant.com/health/hc-swine-flu-pandemic-0612.artjun12,0,3556431.story "Connecticut Records Its Second Swine Flu Death"], by Arielle Levin Becker, The Hartford Courant, June 12, 2009<br> | ||
On June 11, a 6-year-old Connecticut boy died, and his death was "linked" to the H1N1 virus; it was the 2nd recorded Connecticut death attributed to swine flu. This was also the day that the World Health Organization declared swine flu a pandemic.<br> | On June 11, a 6-year-old Connecticut boy died, and his death was "linked" to the H1N1 virus; it was the 2nd recorded Connecticut death attributed to swine flu. This was also the day that the World Health Organization declared swine flu a pandemic.<br> | ||
<blockquote>The boy had underlying medical conditions and had not attended school this year, according to the state Department of Public Health. .... The first person who died, a | <blockquote>The boy had underlying medical conditions and had not attended school this year, according to the state Department of Public Health. .... The first person who died, a ... resident over 50 whose death was announced last week, also had underlying medical problems.</blockquote> | ||
The state health commissioner stated that "this death underscores the seriousness of influenza and the devastating impact it can have." He also said that ordinary seasonal flu kills about 36,000 people a year in the U.S.<br> | The state health commissioner stated that "this death underscores the seriousness of influenza and the devastating impact it can have." He also said that ordinary seasonal flu kills about 36,000 people a year in the U.S.<br> | ||
Line 216: | Line 218: | ||
There is a 1 to 5 scale for measuring pandemics, depending upon "what portion of the population becomes ill and what portion of those with the illness die." The 1918 Spanish flu was a category 5 pandemic, in which "30 percent of the population became ill and 3 percent of those died." It is said to have caused 650,000 deaths in the U.S., and 20-40 million deaths worldwide. The 1968 pandemic was a category 2 pandemic, with 34,000 U.S. deaths ("similar to a typical seasonal flu") and 1 million worldwide deaths.<br> | There is a 1 to 5 scale for measuring pandemics, depending upon "what portion of the population becomes ill and what portion of those with the illness die." The 1918 Spanish flu was a category 5 pandemic, in which "30 percent of the population became ill and 3 percent of those died." It is said to have caused 650,000 deaths in the U.S., and 20-40 million deaths worldwide. The 1968 pandemic was a category 2 pandemic, with 34,000 U.S. deaths ("similar to a typical seasonal flu") and 1 million worldwide deaths.<br> | ||
Discussion<br> | |||
1. What is the difference between a pandemic and an epidemic?<br> | |||
2. Do you agree, or have enough evidence to conclude, that swine flu causes death? Does it appear to be a sufficient and/or necessary condition for mortality?<br> | |||
3. If there were 637 confirmed swine flu cases in Connecticut at the time of the article, how many deaths would you expect in Connecticut, based on the worldwide data given (144 deaths out of 30,000 confirmed cases)? On what assumption(s) would your calculations be based?<br> | |||
4. For the 1918 Spanish flu, estimate the world population based on the information given, that (a) 30% of the population became ill, (b) 3% of those died, and (c) there were 20-40 million deaths worldwide. Is your estimate reasonable, if a U.S. Census Bureau estimate of a worldwide population of 1,860 million in 1920 [http://www.census.gov/ipc/www/worldhis.html] is approximately correct?<br> | |||
5. Why might swine flu mortality be so much lower today than Spanish flu mortality was in 1918?<br> | |||
Submitted by Margaret Cibes<br> | |||
==Swine flu pandemonium II== | |||
[http://www.courant.com/health/hc-swine-flu-death-0616.artjun16,0,7605892,print.story "Third Swine Flu Death Reported In State"], by Mark Spencer, The Hartford Courant, June 16, 2009<br> | |||
A third Connecticut resident, a woman in her 40s with a history of respiratory problems, has died, apparently having contracted swine flu. | |||
<blockquote>As with the state's two previous H1N1 deaths — a person over 50 and a 6-year-old boy — the woman's chronic illness might have weakened her ability to fight the flu. "The trend we've seen so far nationally is anyone who has died had some sort of chronic illness," an official said.</blockquote> | |||
The CT Department of Public Health has revised its confirmed cases figure from to 637 to 693. In the U.S., every state has had a confirmed case of swine flu, with a total of 45 deaths "due to the virus."<br> | |||
Submitted by Margaret Cibes<br> | Submitted by Margaret Cibes<br> | ||
== | ==Meteorite hits boy== | ||
[http://www.telegraph.co.uk/scienceandtechnology/science/space/5511619/14-year-old-hit-by-30000-mph-space-meteorite.html "14-year-old hit by 30,000 mph space meteorite"], ''The Telegraph'', June 12, 2009<br> | |||
<math>\frac{( | Gerrit Blank survived a direct hit to his hand by a meteorite as it hurtled to Earth at "more than 30,000 miles per hour".<br> | ||
<blockquote>A red hot, pea-sized piece of rock then hit his hand before bouncing off and causing a foot wide crater in the ground. | |||
The teenager survived the strike, '''the chances of which are just 1 in a million''' - but with a nasty three-inch long scar on his hand.</blockquote> | |||
From [http://www.wired.com/images_blogs/wiredscience/2009/06/meteorite-nearmisses.jpg Wired magazine], some meteorite "near misses" in history: | |||
http://www.wired.com/images_blogs/wiredscience/2009/06/meteorite-nearmisses.jpg | |||
'''Discussion'''<br> | |||
1. How do you think the speed of 30,000 miles per hour was determined?<br> | |||
2. Is surviving being struck by a meteorite a "1 in a million" chance, or is this rather poetic license? What are the actual probabilities associated with being struck by a meteorite? What about surviving such a strike?<br> | |||
Submitted by Gregory Kohs<br> | |||
==A new record in craps== | |||
[http://www.time.com/time/nation/article/0,8599,1901663,00.html Holy Craps! How a Gambling Grandma Broke the Record] <br> | |||
Time.com, 29 May 2009<br> | |||
Claire Suddath | |||
On May 23, a New Jersey woman named Patricia Demauro set a new world record for the longest turn at craps without "sevening out" by rolling the dice 154 consecutive times. Unfortunately, the article misstates the probability calculation needed to compute the odds of her feat: | |||
<blockquote> | |||
It sounds like a homework problem out of a high school math book: What is the probability of rolling a pair of dice 154 times continuously at a craps table, without throwing a seven? | |||
The answer is roughly 1 in 1.56 trillion... | |||
</blockquote> | |||
It is true that the chance of going 154 (or more) consecutive rolls without rolling a 7 is <math> (\frac{5}{6})^{154}</math>, which is indeed about 1 in 1.56 trillion. Since presumably Patricia failed on the last roll, it would be more accurate to find her chance of rolling more than 153 times. The more serious issue, however, is that "sevening out" is a more complicated event, and refers to rolling a seven after the player has established a "point". | |||
To describe this more fully, we need to review the rules of craps. The player rolling the dice is called the "shooter". The basic bet is called the "Pass" bet, and although many side bets that can be placed during the course of action, it is the Pass bet that governs the play. The shooter begins her turn with an initial roll of the dice. If this is a 2, 3, or 12, the Pass bet loses; if this is a 7 or 11 the pass bet wins; but in either case, the shooter maintains possession of the dice and rolls again. This continues until a 4, 5, 6, 8, 9, or 10 appears, which establishes the shooter's "point". Once a point is established, the game enters a second phase, with the shooter now rolling repeatedly until she either reproduces the point, in which case the Pass bet wins, or else she rolls a seven, in which case the Pass bet loses. The latter option is called "sevening-out", and this is the event upon which the shooter must surrender the dice. In the former case, where the shooter reproduces her point before rolling a 7, she maintains possession of the dice and the whole process starts over. | |||
The initial sequence of rolls prior to establishing a point are called "come-out rolls". It is important to recognize that any number of come-out rolls may produce 7's without ending the shooter's turn with the dice. The flaw in the Time.com analysis was the assumption that any 7 would end the turn. The more complicated process of sevening out can be modeled using an absorbing Markov chain, with state space {0,1,2,3,4} defined by | |||
<center> | |||
{| border="0" | |||
|- | |||
! 0: | |||
| come out rolls | |||
|- | |||
! 1: | |||
| point is 4 or 10 | |||
|- | |||
! 2: | |||
|point is 5 or 9 | |||
|- | |||
! 3: | |||
| point is 6 or 8 | |||
|- | |||
! 4: | |||
| sevened out | |||
|- | |||
|} | |||
</center> | |||
The chain is initially in state 0. For concreteness, here is a possible realization of the process, showing the player's roll and the corresponding state of the chain. | |||
<center> | |||
{| border="1" style="text-align:center" | |||
|- | |||
|width="25%"| ''n'' | |||
|width="5%"| 0 ||width="5%"| 1 ||width="5%"| 2 ||width="5%"| 3 | |||
|width="5%"| 4||width="5%"| 5||width="5%"| 6||width="5%"| 7 | |||
|width="5%"| 8||width="5%"| 9||width="5%"| 10||width="5%"| 11 | |||
|width="5%"| 12 | |||
|- | |||
| ''nth'' roll | |||
| _ || 2 || 7 || 11 || 5 || 8 || 12 || 4 || 5 || 7 || 4 || 10 || 7 | |||
|- | |||
| state of chain | |||
| 0 || 0 || 0 || 0 || 2 || 2 || 2 || 2 || 0 || 0 || 1 || 1 || 4 | |||
|- | |||
|} | |||
</center> | |||
In this illustration, the shooter first establishes a point, namely 5, on the fourth roll. This moves the chain to state 2, where it remains until the shooter reproduces a the 5 on the eighth roll. This starts another sequence of come-out attempts. A point of 4 is established on the tenth roll, and the shooter sevens-out on the twelfth roll, ending her turn. | |||
The probability transition matrix ''P'' for the Markov chain is given below: | |||
<center> | |||
{| border="1" style="text-align:center" cellpadding="5" | |||
|+ Transition Matrix | |||
! !! 0 !! 1 !! 2 !! 3 !! 4 | |||
|- | |||
! 0 | |||
| 12/36 || 6/36 || 8/36 || 10/36 || 0 | |||
|- | |||
! 1 | |||
| 3/36 || 27/36 || 0 || 0 || 6/36 | |||
|- | |||
! 2 | |||
| 4/36 || 0 || 26/36 || 0 || 6/36 | |||
|- | |||
! 3 | |||
| 5/36 || 0 || 0 || 25/36 || 6/36 | |||
|- | |||
! 4 | |||
| 0 || 0 || 0 || 0 || 1 | |||
|- | |||
|} | |||
</center> | |||
The sevened-out state is absorbing; hence the form of row 4. The probabilities for the other rows are easily computed. From state 0, the chance of rolling 2, 3, 7, 11, or 12 is (1+2+6+2+1)/36 = 12/36 which keeps the chain in state 0. The chance of rolling 4 or 10 is (3+3)/36 which leads to state 1; similar calculations hold for transitions to states 2 and 3. Next, from state 1, the chance of reproducing the point is 3/36, which leads to state 0; the chance of rolling a 7 is 6/36 which leads to state 4; otherwise the chain remains in state 1. Rows 2 and 3 are analogous to row 1. | |||
To analyze the chain, we use standard absorbing chain theory, following notation from Grinstead and Snell's [http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/book.html Introduction to Probability] text. (Concise notes from a recent Dartmouth course are [http://www.math.dartmouth.edu/archive/m20x06/public_html/Lecture14.pdf here]). The leading 4x4 submatrix corresponding to the transient states is denoted <math>Q</math>. The probabilities of still being in particular transient states after 153 rolls is given by entries in the vector <math>(1,0,0,0)Q^{153}</math>, and the sum of these probabilities is the chance of not having sevened out after 153 rolls. It is easy to implement this computation iteratively, giving <math>1.788824 \times 10^{-10}</math>, which approximately 1 in 5.59 billion. This certainly is a small probability, but much larger than the 1 in 1.56 trillion from Time.com. | |||
You can find a discussion of this problem in [http://blogs.wsj.com/numbersguy/crunching-the-numbers-on-a-craps-record-703/ Crunching the numbers on a craps record] from Carl Bialik's Wall Street Journal column, "The Numbers Guy". He explains why the sevening-out event is complicated, and reports on various attempts to solve the problem. In the final update, he reports that Keith Crank of the American Statistical Association used a Markov chain analysis (which presumably corresponds to what we did above) to find a probability of 1 in 5.6 billion. Earlier in the article he cites simulation results from Professor Michael Shackleford, referencing Shackleford's [http://wizardofodds.com/askthewizard/askcolumns/askthewizard81.html Wizard of Odds] website. Apart from the simulation results, Shackleford presents a recursive computation scheme that is equivalent to the Markov chain. You can also read there interesting historical notes on craps records. | |||
Returning to the Time.com piece, we read "The average number of dice rolls before sevening out? Eight." The true value is about 8.5, but it doesn't follow from their original analysis, which asked only how long it takes to roll a seven (the expected number of rolls is 6). Using the Markov chain model, the expected time to absorption is the sum of the row zero entries in the fundamental matrix <math>N = (I-Q)^{-1}</math>. Computing this gives an expected 8.5 rolls to seven-out. | |||
Time also interviewed Professor Thomas Cover of Stanford, who pointed out that while the probability of the event in question is small, one needs to remember that there are many people playing craps at any time, all of whom are in principle contending for the record! Shackleford estimates their are about 50 million craps turns per year in the US, giving about a 1% chance that a feat like Demauro's would occur in a given year. | |||
DISCUSSION QUESTION:<br> | |||
How do you think Shackleford arrived at the 50 million estimate? Can you use the earlier post on Guesstimating to come up with a figure? | |||
Submitted by Bill Peterson |
Latest revision as of 15:53, 29 August 2018
Quotations
Probability arises from an opposition of contrary chances or causes, by which the mind is not allowed to fix on either side, but is incessantly tost [sic] from one to another, and at one moment is determined to consider an object as existent, and at another moment as the contrary.
Submitted by Margaret Cibes
After reading the above, I coincidentally came across this David Hume quotation on page 72 of Gould's The Mismeasure of Man, second edition:
I am apt to suspect the negroes and in general all the other species of men...to be naturally inferior to the whites. There never was a civilized nation of any other complexion than white, nor an individual eminent either in action or speculation. No ingenious manufacturers amongst them, no arts, no sciences.
The quotation continues on in the same vein and makes one's head spin at the attitudes and prejudices of the famous philosophers.
Submitted by Paul Alper
Forsooths
Steven J. Dubner of the New York Times writes about Bernice Geiger, a person who "never took vacations" for fear of her embezzlement being discovered by a fill-in employee; she "was arrested in 1961 for embezzling more than $2 million over the course of many years." Eventually, "after prison Geiger went to work for a banking oversight agency to help stop embezzlement."
Geiger's "biggest contribution: looking for employees who failed to take vacation. This simple metric turned out to have strong predictive power in stopping embezzlement."
Submitted by Paul Alper
Infuse and Kuklo II
This web site provides a wonderful pun regarding Benford’s Law, “Looking out for number one.” The authors write: “Go and look up some numbers. A whole variety of naturally-occurring numbers will do. Try the lengths of some of the world's rivers, or the cost of gas bills in Moldova; try the population sizes in Peruvian provinces, or even the figures in Bill Clinton's tax return. Then, when you have a sample of numbers, look at their first digits (ignoring any leading zeroes). Count how many numbers begin with 1, how many begin with 2, how many begin with 3, and so on - what do you find? You might expect that there would be roughly the same number of numbers beginning with each different digit: that the proportion of numbers beginning with any given digit would be roughly 1/9. However, in very many cases, you'd be wrong!”
Instead, we get
Should somebody try “to falsify, say, their tax return then invariably they will have to invent some data. When trying to do this, the tendency is for people to use too many numbers starting with digits in the mid range, 5,6,7 and not enough numbers starting with 1. This violation of Benford's Law sets the alarm bells ringing.”
It is a pity that unlike for accounting data, there is no forensic counterpart to Benford’s Law for determining when a journal article is entirely fraudulent. As stated in Infuse and Kuklo you won’t be able to read [on the JBJS website] the fraudulent article, “Recombinant human morphogenetic protein-2 for type grade III open segmental tibial fractures from combat injuries in Iraq” by Timothy Kuklo, et al, which appeared in the JBJS in August, 2008 because it has been retracted. However, it is available here. The immediate impression is that as far as statistics is concerned, it looks like any other article in the health field.
The important statistics appear in Tables 1 and III
http://www.dartmouth.edu/~chance/forwiki/Table1.jpg
http://www.dartmouth.edu/~chance/forwiki/Table3.gif
Note that there is no claim that everyone in Group 2 (the group using Infuse) did well or that everyone in Group 1 fared poorly. Further, as in legitimate studies, there are patients who were not included because of an additional problem (head injury) or were lost to follow up. The data is there for reviewers and others to do the calculations which in this paper are the difference in proportions, a standard statistical technique. Small but not immodest p-values indicate that statistical significance is obtained; detailed discussion about the fractures indicates that practical significance is also realized. The bibliography has 39 entries, only one of which has Kuklo as the author; the same entry includes one of the ghost co-authors in the retracted paper. Nothing statistically or otherwise suspicious whatsoever.
Freudian psychology is currently out of favor but Freud's notion of a death wish still seems plausible. How else to explain the pushing of the envelope past falsification of data, denial of connection to the manufacturers of Infuse, and forging of not one, not two but four ghost authors? The aptly titled 1995 book by Feinberg and Tarrant, Why Smart People Do Dumb Things, attributes such behavior to what they deem “the four pillars of stupidity”: hubris, arrogance, narcissism and unconscious need to fail. The first three are overwhelmingly obvious, but the last named cause sounds deeply Freudian.
A New York Times update appears on June 5, 2009 and shows how Kuklo forged the signatures; “He used a distinctively different handwriting style for each of them, a form he submitted to the British journal shows.”
http://www.dartmouth.edu/~chance/forwiki/Kuklosignatures.jpg
Dr. Timothy R. Kuklo and copies of the signatures of other Army doctors on his study that authorities say he forged.
A putative co-author “suspected that Dr. Kuklo had fabricated the comparison groups, because many soldiers had received both Infuse and a bone graft — not one or the other.” This person said, “It was like he was comparing apples and oranges. But there weren’t any apples or oranges to compare.”
Returning to the statistical aspect of the paper, Table III says 19 of 67 (28%) in Group 1 were patients who had further surgery while 5 of 62 (8%) in Group 2 (Infuse group) had further surgery. Presumably, via a chi-square test, the p-value is listed as .003. Minitab produces the same numerical result of .003 via the Fisher exact test:
Sample X N Sample p
1 5 62 0.080645
2 19 67 0.283582
Difference = p (1) - p (2)
Estimate for difference: -0.202937
95% CI for difference: (-0.330382, -0.0754923)
Test for difference = 0 (vs not = 0): Z = -3.12 P-Value = 0.002
Fisher's exact test: P-Value = 0.003
Some numerical discrepancies arise, however, for Table I. Table I says 51 of 67 (76%) in Group 1 had a successful “union” while 57 of 62 (92%) in Group 2 (Infuse group) had a successful union. Presumably, via a chi-square test, the p-value is listed as .015. Minitab produces the following indicating that because of the small sample sizes, the Fisher exact test yields .017 instead:
Sample X N Sample p
1 57 62 0.919355
2 51 67 0.761194
Difference = p (1) - p (2)
Estimate for difference: 0.158161
95% CI for difference: (0.0356210, 0.280701)
Test for difference = 0 (vs not = 0): Z = 2.53 P-Value = 0.011
Fisher's exact test: P-Value = 0.017
Table I also says 10 of 67 (14%) in Group 1 had post-operative infections while 2 of 62 (3.2%) in Group 2 (Infuse group) had post-operative infections. Presumably, via a chi-square test, the p-value is listed as .001. Minitab produces the following quite different p-value of .032:
Sample X N Sample p
1 10 67 0.149254
2 2 62 0.032258
Difference = p (1) - p (2)
Estimate for difference: 0.116996
95% CI for difference: (0.0210037, 0.212988)
Test for difference = 0 (vs not = 0): Z = 2.39 P-Value = 0.017
Fisher's exact test: P-Value = 0.032
However, these discrepancies are hardly in the Benford class. They may merely indicate what happens when a non-statistician medical doctor acts alone.
Submitted by Paul Alper
For nice video about Benford's Law, see online lecture by Mark Nigrini, of the Cox School of Business [1], from the Chance Lecture Series 2000.
Submitted by Margaret Cibes
Emotional biases in financial decisions
"Control Yourself", by Veronica Dagher, The Wall Street Journal, June 8, 2009
This article describes 5 "biases", or emotional issues, that affect investment decisions and that are studied in the field of "behavioral finance."
(1) "Anchoring" bias refers to being "overly attached to a particular investment."
(2) "Recency" bias refers to assuming that "events or patterns in the past will continue into the future."
(3) "Loss aversion" bias refers to "hoping inaction [will] eventually make the losses go away."
(4) "Endowment effect" bias refers to assigning a "greater value to what [one] own[s] than to what [one doesn't] own, whether that value is warranted or not."
(5) "Overconfidence" bias refers to excessive trading in an attempt to "beat the market."
“One could try to explain all the events of the last several months with models and ratios, but it’s become more and more difficult to do so,” says Richard Thaler, professor of behavioral science and economics at the Booth School of Business at the University of Chicago.
Submitted by Margaret Cibes
Guesstimation
The biggest of puzzles brought down to size.
by Natalie Angier, New York Times, 30 March 2009
The article opens by reminding us that with bank bailouts running hundreds of billions of dollars the national debt passing ten trillion, the public need help comparing the magnitudes of really large numbers. For practice, the author recommends so-called "Fermi problems,". Named for Enrico Fermi, these are estimation problems that physicists and engineers like to use to sharpen their intuition. Two examples cited in the article are:
What is the total volume of human blood in the world? or, If you put all the miles that Americans drive every year end to end, how far into space could you travel?
Readers may recall that a number of such problems were described by John Allen Paulos in his classic, Innumeracy, where he lamented the fact that estimation skills were not being taught in the schools. A more recent source, featured in the present article, is Guesstimation: Solving the World’s Problems on the Back of a Cocktail Napkin (Princeton University Press, 2008). A companion article on 31 March gives an online quiz based on the book.
The book serves as the text for course at Princeton in Spring, 2009, "Physics 309: Physics on the Back of an Envelope", which was offered by one of the book's co-authors, Professor Lawrence Weinstein. His course website links to sample midterms and to similar courses at MIT and CalTech.
Submitted by Bill Peterson
Measuring drivers' drunken-ness
"Drunk Driver Data Don't Walk Straight Line Either", by Carl Bialik (The Numbers Guy), The Wall Street Journal, June 10, 2009
This article describes a disagreement between Mothers Against Drunk Driving and a liquor-industry-funded group, Century Council, about the blood alcohol level that would trigger a proposed penalty requiring convicted drunk drivers to install an ignition interlock to prevent them driving when their breath alcohol level is "too high."
Century Council has stated that it wants to limit the level to a minimum of 0.15 grams per deciliter of blood, based on 2007 government studies that show that 3 out of 5 drivers involved in alcohol-related fatal crashes had a BACof at least 0.15, in contrast to about 1 out of 5 with BACs of 0.01 to 0.08 and 1 out of 5 with BACs of 0.09 to 0.14.
According the article's author, a Weststat statistician believes that "the same personality traits that lead to driving while highly intoxicated are probably tied to other risky behavior behind the wheel" and that
these heavy drinkers are far more dangerous than other drunken drivers on the road. [He] compared the blood-alcohol levels of drivers killed in crashes with levels of drivers stopped for random roadside testing during peak drunken-driving hours. .... Compared with sober drivers, drivers at 0.15 or higher were about 400 times more likely to die in a crash. Drivers with levels between 0.10 and 0.14 were 50 times more likely than sober drivers to die in a crash.
MADD prefers a minimum "high" that is the legal limit of 0.08. A 2002 study at Johns Hopkins University, based on interviews with surviving family members of over 800 victims of fatal crashes, found that 55% of dead drivers with BAC levels of 0.15 or higher, and 35% of those with BAC levels between 0.10 and 0.14, drank at least monthly, leading a study co-author to state, "We shouldn't simply be focusing on 'hard-core' drivers."
According to the article's author, "some researchers would prefer to see a lower limit, with penalties tied to the blood-alcohol level, like with speeding penalties. .... Complicating matters, people's alcohol-metabolism varies, as does the relationship between their breath alcohol ... and their blood alcohol."
A blogger wrote, "I recall several years ago, a drunk was let go free because he was able to prove the variability in the gage [sic]."
See "The Numbers All Drivers Should Know" for more information on this topic from The Numbers Guy.
Submitted by Margaret Cibes
Variables lurk in Wal-Mart study
"Wal-Mart's Weight Effect", by Art Carden, Forbes Magazine, June 8, 2009
This story reports preliminary findings from a University of NC-Greensboro study of big retail stores and obesity. The author of the article is a co-author of the study.
In [the] first round of statistical analysis we found that greater consumer access to a Wal-Mart ... store was associated with lower body-mass indexes and a lower probability of being obese. ... [T]he correlation holds up under a variety of different circumstances, with a clear relationship between warehouse clubs and better eating habits emerging over time. Further, ... Wal-Mart's effect on weight is largest for women, the poor, African-Americans and people who live in urban areas. .... [W]hile we found a statistically significant effect on body mass index, the effect is very, very small.
One blogger suggests that the observed effect of big retail stores on obesity may be a result of the fact that shoppers who purchase fresh fruits and vegetables at stores like Costco have to eat lots of these healthy foods in shorter periods of time because the packages are very large and the contents are perishable.
A second blogger writes, "I notice that people who live within a 2-3 mile radius of my local Wal-Mart are better educated, have better access to health care (... a hospital), have more parks in close proximity, join more adult softball teams, and probably go to the dentist more often.
.... This is a correlation [that] has to do with where Wal-Mart locates stores."
A third blogger suggests an "exercise effect" due to long walks through large parking lots for large retail stores.
Submitted by Margaret Cibes
A New Math War?
The Chronicle of Higher Education June 12, 2009 Jeffrey R. Young
This article suggests that Wolfram's new WolframAlpha will create a war over whether their calculus students should be allowed to have WolframAlpha solve their homework. Of course their is nothing special about calculus because Alpha can also solve problems in other math courses, for example statistics. In addition to giving the answer to a problem but Alpha also tells how it found the solution. There is disagreement about whether they should allow the students to use Alpha, but it does not seem that it will lead to war as suggested by the title.
Those who do not want to change their way of teaching will probably say that students cannot use Alpha while those who are willing to change their ways will figure out a way to take advantage of their students use Alpha. Or they can follow the advice of David Bressoud, president of the Mathematical Association of America who says:
Most math instructors now realize that the end-all and be-all of math instruction is not to give students algorithmic facility, but it really is to understand the mathematical ideas and understand how to use them.
Submitted by Laurie Snell
Lead "paint"?
"A Simple Smooch or a Toxic Smack?", by Abby Ellin, The New York Times, May 28, 2009
This article discusses concerns about lead content in lipsticks. Some doctors and others believe that lipsticks contain high levels of lead, while the FDA believes that any lead content would merely be a harmless trace. Doctors also disagree about whether there is any "safe" level of lead.
In 2007 a study [2] by a citizens' advocacy group, Campaign for Safe Cosmetics, found that "one-third of 33 lipsticks had lead in excess of 0.1 parts per million, the federal limit for candy."
Among the worst offenders were L’Oreal Colour Riche “True Red” lipstick (with a lead content of 0.65 parts per million) and Cover Girl’s Incredifull Lipcolor “Maximum Red” (0.56 p.p.m.). Price had nothing to do with lead levels: less expensive brands, like a $1.99 tube of Wet and Wild Mega Colors “Cherry Blossom,” contained no lead, whereas a $24 tube of Dior Addict “Positive Red” [since discontinued] contained 0.21 p.p.m.
Manufacturers claim that their cosmetics are safe because they satisfy FDA requirements: manufacturers are only required to list "intended ingredients," not "unintended byproducts" of a manufacturing process, such as lead. Nevertheless, the advocacy group wants the FDA to release its data and to set a safety standard for lead in lipstick, not wait for a "peer-reviewed journal to publish its study of lead in lipstick."
The editor of Stats, at George Mason University's Center for Health and Risk Communication, is quoted in the article: "These things sound terribly scary, but there’s a massive disconnect between how toxicologists evaluate risks and how activist groups evaluate risk, and even then there are debates.” In a March survey of over 900 members of the Society of Toxicology, 66% disagreed that cosmetics are a "significant source of chemical health risk," while 26% agreed and 8% "didn't know."
Submitted by Margaret Cibes
Swine flu pandemonium I
"Connecticut Records Its Second Swine Flu Death", by Arielle Levin Becker, The Hartford Courant, June 12, 2009
On June 11, a 6-year-old Connecticut boy died, and his death was "linked" to the H1N1 virus; it was the 2nd recorded Connecticut death attributed to swine flu. This was also the day that the World Health Organization declared swine flu a pandemic.
The boy had underlying medical conditions and had not attended school this year, according to the state Department of Public Health. .... The first person who died, a ... resident over 50 whose death was announced last week, also had underlying medical problems.
The state health commissioner stated that "this death underscores the seriousness of influenza and the devastating impact it can have." He also said that ordinary seasonal flu kills about 36,000 people a year in the U.S.
Worldwide, there have been nearly 30,000 confirmed cases of the H1N1 virus in 74 countries. Nearly half of the confirmed cases were reported in the U.S. .... So far, swine flu has caused 144 deaths worldwide, compared with ordinary flu, which kills up to 500,000 people a year. .... In Connecticut, there have been 637 confirmed cases of swine flu, though health officials say the number of cases is likely much higher.
The World Health Organization director-general stated that "the overwhelming majority of patients experience mild symptoms and make a rapid and full recovery, often in the absence of any form of medical treatment." According to the article's author, "The pandemic designation refers to the virus's sustained geographic spread, not its severity."
There is a 1 to 5 scale for measuring pandemics, depending upon "what portion of the population becomes ill and what portion of those with the illness die." The 1918 Spanish flu was a category 5 pandemic, in which "30 percent of the population became ill and 3 percent of those died." It is said to have caused 650,000 deaths in the U.S., and 20-40 million deaths worldwide. The 1968 pandemic was a category 2 pandemic, with 34,000 U.S. deaths ("similar to a typical seasonal flu") and 1 million worldwide deaths.
Discussion
1. What is the difference between a pandemic and an epidemic?
2. Do you agree, or have enough evidence to conclude, that swine flu causes death? Does it appear to be a sufficient and/or necessary condition for mortality?
3. If there were 637 confirmed swine flu cases in Connecticut at the time of the article, how many deaths would you expect in Connecticut, based on the worldwide data given (144 deaths out of 30,000 confirmed cases)? On what assumption(s) would your calculations be based?
4. For the 1918 Spanish flu, estimate the world population based on the information given, that (a) 30% of the population became ill, (b) 3% of those died, and (c) there were 20-40 million deaths worldwide. Is your estimate reasonable, if a U.S. Census Bureau estimate of a worldwide population of 1,860 million in 1920 [3] is approximately correct?
5. Why might swine flu mortality be so much lower today than Spanish flu mortality was in 1918?
Submitted by Margaret Cibes
Swine flu pandemonium II
"Third Swine Flu Death Reported In State", by Mark Spencer, The Hartford Courant, June 16, 2009
A third Connecticut resident, a woman in her 40s with a history of respiratory problems, has died, apparently having contracted swine flu.
As with the state's two previous H1N1 deaths — a person over 50 and a 6-year-old boy — the woman's chronic illness might have weakened her ability to fight the flu. "The trend we've seen so far nationally is anyone who has died had some sort of chronic illness," an official said.
The CT Department of Public Health has revised its confirmed cases figure from to 637 to 693. In the U.S., every state has had a confirmed case of swine flu, with a total of 45 deaths "due to the virus."
Submitted by Margaret Cibes
Meteorite hits boy
"14-year-old hit by 30,000 mph space meteorite", The Telegraph, June 12, 2009
Gerrit Blank survived a direct hit to his hand by a meteorite as it hurtled to Earth at "more than 30,000 miles per hour".
A red hot, pea-sized piece of rock then hit his hand before bouncing off and causing a foot wide crater in the ground. The teenager survived the strike, the chances of which are just 1 in a million - but with a nasty three-inch long scar on his hand.
From Wired magazine, some meteorite "near misses" in history:
http://www.wired.com/images_blogs/wiredscience/2009/06/meteorite-nearmisses.jpg
Discussion
1. How do you think the speed of 30,000 miles per hour was determined?
2. Is surviving being struck by a meteorite a "1 in a million" chance, or is this rather poetic license? What are the actual probabilities associated with being struck by a meteorite? What about surviving such a strike?
Submitted by Gregory Kohs
A new record in craps
Holy Craps! How a Gambling Grandma Broke the Record
Time.com, 29 May 2009
Claire Suddath
On May 23, a New Jersey woman named Patricia Demauro set a new world record for the longest turn at craps without "sevening out" by rolling the dice 154 consecutive times. Unfortunately, the article misstates the probability calculation needed to compute the odds of her feat:
It sounds like a homework problem out of a high school math book: What is the probability of rolling a pair of dice 154 times continuously at a craps table, without throwing a seven? The answer is roughly 1 in 1.56 trillion...
It is true that the chance of going 154 (or more) consecutive rolls without rolling a 7 is <math> (\frac{5}{6})^{154}</math>, which is indeed about 1 in 1.56 trillion. Since presumably Patricia failed on the last roll, it would be more accurate to find her chance of rolling more than 153 times. The more serious issue, however, is that "sevening out" is a more complicated event, and refers to rolling a seven after the player has established a "point".
To describe this more fully, we need to review the rules of craps. The player rolling the dice is called the "shooter". The basic bet is called the "Pass" bet, and although many side bets that can be placed during the course of action, it is the Pass bet that governs the play. The shooter begins her turn with an initial roll of the dice. If this is a 2, 3, or 12, the Pass bet loses; if this is a 7 or 11 the pass bet wins; but in either case, the shooter maintains possession of the dice and rolls again. This continues until a 4, 5, 6, 8, 9, or 10 appears, which establishes the shooter's "point". Once a point is established, the game enters a second phase, with the shooter now rolling repeatedly until she either reproduces the point, in which case the Pass bet wins, or else she rolls a seven, in which case the Pass bet loses. The latter option is called "sevening-out", and this is the event upon which the shooter must surrender the dice. In the former case, where the shooter reproduces her point before rolling a 7, she maintains possession of the dice and the whole process starts over.
The initial sequence of rolls prior to establishing a point are called "come-out rolls". It is important to recognize that any number of come-out rolls may produce 7's without ending the shooter's turn with the dice. The flaw in the Time.com analysis was the assumption that any 7 would end the turn. The more complicated process of sevening out can be modeled using an absorbing Markov chain, with state space {0,1,2,3,4} defined by
0: | come out rolls |
---|---|
1: | point is 4 or 10 |
2: | point is 5 or 9 |
3: | point is 6 or 8 |
4: | sevened out |
The chain is initially in state 0. For concreteness, here is a possible realization of the process, showing the player's roll and the corresponding state of the chain.
n | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 |
nth roll | _ | 2 | 7 | 11 | 5 | 8 | 12 | 4 | 5 | 7 | 4 | 10 | 7 |
state of chain | 0 | 0 | 0 | 0 | 2 | 2 | 2 | 2 | 0 | 0 | 1 | 1 | 4 |
In this illustration, the shooter first establishes a point, namely 5, on the fourth roll. This moves the chain to state 2, where it remains until the shooter reproduces a the 5 on the eighth roll. This starts another sequence of come-out attempts. A point of 4 is established on the tenth roll, and the shooter sevens-out on the twelfth roll, ending her turn.
The probability transition matrix P for the Markov chain is given below:
0 | 1 | 2 | 3 | 4 | |
---|---|---|---|---|---|
0 | 12/36 | 6/36 | 8/36 | 10/36 | 0 |
1 | 3/36 | 27/36 | 0 | 0 | 6/36 |
2 | 4/36 | 0 | 26/36 | 0 | 6/36 |
3 | 5/36 | 0 | 0 | 25/36 | 6/36 |
4 | 0 | 0 | 0 | 0 | 1 |
The sevened-out state is absorbing; hence the form of row 4. The probabilities for the other rows are easily computed. From state 0, the chance of rolling 2, 3, 7, 11, or 12 is (1+2+6+2+1)/36 = 12/36 which keeps the chain in state 0. The chance of rolling 4 or 10 is (3+3)/36 which leads to state 1; similar calculations hold for transitions to states 2 and 3. Next, from state 1, the chance of reproducing the point is 3/36, which leads to state 0; the chance of rolling a 7 is 6/36 which leads to state 4; otherwise the chain remains in state 1. Rows 2 and 3 are analogous to row 1.
To analyze the chain, we use standard absorbing chain theory, following notation from Grinstead and Snell's Introduction to Probability text. (Concise notes from a recent Dartmouth course are here). The leading 4x4 submatrix corresponding to the transient states is denoted <math>Q</math>. The probabilities of still being in particular transient states after 153 rolls is given by entries in the vector <math>(1,0,0,0)Q^{153}</math>, and the sum of these probabilities is the chance of not having sevened out after 153 rolls. It is easy to implement this computation iteratively, giving <math>1.788824 \times 10^{-10}</math>, which approximately 1 in 5.59 billion. This certainly is a small probability, but much larger than the 1 in 1.56 trillion from Time.com.
You can find a discussion of this problem in Crunching the numbers on a craps record from Carl Bialik's Wall Street Journal column, "The Numbers Guy". He explains why the sevening-out event is complicated, and reports on various attempts to solve the problem. In the final update, he reports that Keith Crank of the American Statistical Association used a Markov chain analysis (which presumably corresponds to what we did above) to find a probability of 1 in 5.6 billion. Earlier in the article he cites simulation results from Professor Michael Shackleford, referencing Shackleford's Wizard of Odds website. Apart from the simulation results, Shackleford presents a recursive computation scheme that is equivalent to the Markov chain. You can also read there interesting historical notes on craps records.
Returning to the Time.com piece, we read "The average number of dice rolls before sevening out? Eight." The true value is about 8.5, but it doesn't follow from their original analysis, which asked only how long it takes to roll a seven (the expected number of rolls is 6). Using the Markov chain model, the expected time to absorption is the sum of the row zero entries in the fundamental matrix <math>N = (I-Q)^{-1}</math>. Computing this gives an expected 8.5 rolls to seven-out.
Time also interviewed Professor Thomas Cover of Stanford, who pointed out that while the probability of the event in question is small, one needs to remember that there are many people playing craps at any time, all of whom are in principle contending for the record! Shackleford estimates their are about 50 million craps turns per year in the US, giving about a 1% chance that a feat like Demauro's would occur in a given year.
DISCUSSION QUESTION:
How do you think Shackleford arrived at the 50 million estimate? Can you use the earlier post on Guesstimating to come up with a figure?
Submitted by Bill Peterson