Sandbox: Difference between revisions

From ChanceWiki
Jump to navigation Jump to search
No edit summary
 
Line 1: Line 1:
<table>
|              | Treatment|                                              Deaths per 1000 women|
|No mammography screening|4|
|Mammography screening|3|                                                       
</table>                                                           


Consequently, there is "a 25 percent relative risk reduction."  He would prefer focusing on the difference in the number of deaths which yields the more revealing and perhaps more honest statement: "The absolute risk reduction is 4 minus 3, that is, 1 out of 1000 women (which corresponds to .1 percent)."  However, "Counting on their clients' innumeracy, organizations that want to impress upon clients the benefits of treatment generally report them in terms of relative risk reduction...applicants [for grants] often feel compelled to report relative risk reductions because they sound more impressive."  Although he did not use this example, one's relative "risk" of winning the lottery is infinitely greater if one buys a ticket, yet one's absolute "risk" of winning has hardly improved at all.
Most of his numerical examples are typified by his discussion of the picture given below which indicates the superiority of dealing with counts.  Note that "H"  represents having the disease and "D" represents a diagnosis having the symptom as seen by testing positive.  Characteristically, there is a large number in the population who do not have the disease and because of the possibility of a wrong classification, the number of false positives (99) outweighs the number of true positives (8) resulting in P(disease| symptom) being much lower (8/(8+99)) than P(symptom| disease) (.8).  This type of result, low probability of disease given symptom, is true even when ".8" is replaced by a number much closer to 1 provided there are many more who do not have the disease.
Here is an example he did not consider but it also illustrates the superiority of dealing with counts.  Instead of two populations--diseased and healthy--which are greatly different in size, consider Boys and Girls and the desire to predict gender based on some simple test.  Assume that 50% of births are Boys so that P(Boy) = P(Girl) = 1/2.  A simple, inexpensive, non-invasive gender-testing procedure indicates that it is "perfect" for boys, P(Test Boy| Boy) = 1, implying P(Test Girl| Boy) = 0.  Unfortunately, this simple, inexpensive, non-invasive gender-testing procedure for girls is a "coin toss," P(Test Girl| Girl) = P(Test Boy| Girl) = 1/2.  Application of Bayes theorem yields what seems to be a strange inversion, P(Boy| Test Boy) = 2/3 and P(Girl| Test Girl) = 1.  That is, somehow, "perfection" switched from Boy to Girl.  The test is perfect in "confirming" that a Boy is a Boy and has a 50% error rate in confirming that a Girl is a Girl.  The test is perfect in "predicting" that a person who tests as a girl is in fact a girl but has 33% error rate in predicting that a person who tests as a Boy is in fact a Boy.  Thus, the term perfect is ambiguous.  Perfection in confirmation, i.e., the test conditional on the gender, does not mean perfection in prediction, i.e., the gender conditional on the test.
                             


Some of the  puzzlement disappears if we deal with counts; the table below is equivalent to Gigerenzer's "tree" diagram.  Assume 50 Boys and 50 Girls to start with.  Every one of the 50 Boys will test as a Boy--none of the Boys test as a Girl; of the 50 Girls, 25 will test as a Boy and 25 will test as a girl.  Therefore, P(Girl| Test Girl) = 1.  One is tempted to to explain the switch by using the lingo of medical testing: false positives, false negatives, sensitivity, specificity, positive predictive value, negative predictive value.  However, one hesitates to designate either gender as diseased even though the mathematics is the same.
==Forsooth==
                                Test Boy                  Test Girl              Total


Boy                            50                              0                      50
==Quotations==
“We know that people tend to overestimate the frequency of well-publicized, spectacular
events compared with more commonplace ones; this is a well-understood phenomenon in
the literature of risk assessment and leads to the truism that when statistics plays folklore,
folklore always wins in a rout.”
<div align=right>-- Donald Kennedy (former president of Stanford University), ''Academic Duty'', Harvard University Press, 1997, p.17</div>


            Girl                            25                            25                      50
----
Gigerenzer rightly concludes that the language of statistics is not natural for most individuals.  Perhaps the puzzlement in this specific example is at least partly due to the natural language known as English.  Boys, Girls, Test Boys and Test Girls are too confusing.  .  Replace "Boy" by "Norwegian" and "Girl" by "German" and assume that there are as many Norwegians as Germans.  Let every Norwegian be "Blond,"  so that P(Blond| Norwegian) = 1 and only half the Germans are Blond.  Thus, P(German| Not Blond) =1; the switch, P(German| Not Blond) = P(Blond| Norwegian) = 1, is rather obvious.  Is the this situation easier to understand because of the linguistics--hair color and ethnicity are easily distinct as Test Boy and Boy are not?


Discussion Questions
"Using scientific language and measurement doesn’t prevent a researcher from conducting flawed experiments and drawing wrong conclusions — especially when they confirm preconceptions."
1. Gigerenzer has a chapter entitled, "(Un)Informed Consent." Based on your experience, what do you imagine the chapter contains?
 
2. A drawing of two tables (that is, physical tables on which things are placed) appears on page 10.  He claims the tables (due to Roger Shepard) are identical in size and shape. After staring at them in disbelief of the claim, how would you verify the contention?
<div align=right>-- Blaise Agüera y Arcas, Margaret Mitchell and Alexander Todoorov, quoted in: The racist history behind facial recognition, ''New York Times'', 10 July 2019</div>
3. Physicians sometime make the following type of statement:"Never mind the statistics, I treat every patient as an individual." Defend this assertion.  Criticize this assertion.
 
4. The physicist, Lord Rutherford, is reputed to have said, " If your experiment, needs statistics you ought to have done a better experiment."  Defend and criticize Lord Rutherford.
==In progress==
5. Assume an asymptomatic woman has a mammogram which looks suspicious and then a biopsy which is negative.  Would she be grateful for the clean bill of health or would she become an advocate who opposes (mass) screening? Suppose instead we assume a man has a suspiciously high PSA and the painful multiple biopsies (6-12 "sticks") are all negative.   Would he be grateful for the clean bill of health or would he become an advocate who opposes (mass) screening?  
[https://www.nytimes.com/2018/11/07/magazine/placebo-effect-medicine.html What if the Placebo Effect Isn’t a Trick?]<br>
6. Calculated Risks also deals with the risk to the physician making a recommendation and a diagnosis.  Discuss why in our present-day litigious society the risks to the physician (who may or may not recommend a test or may or may not make a diagnosis)  are not symmetrical.  Along these lines, who are the vested interests involved in maintaining screening and testing?
by Gary Greenberg, ''New York Times Magazine'', 7 November 2018
7.  Revisit the Boy/Girl scenario but now the test always says Boy regardless of gender, P(Test Boy| Boy) = P(Test Boy| Girl) = 1. Complete the table for this version. Obviously, this test has the advantage of being extremely simple, cost-free and non-invasive.  Use either the Probability Format or the Frequency Format to comment on the statistical worthiness of this test.
 
[https://www.nytimes.com/2019/07/17/opinion/pretrial-ai.html The Problems With Risk Assessment Tools]<br>
by Chelsea Barabas, Karthik Dinakar and Colin Doyle, ''New York Times'', 17 July 2019
 
==Hurricane Maria deaths==
Laura Kapitula sent the following to the Isolated Statisticians e-mail list:
 
:[Why counting casualties after a hurricane is so hard]<br>
:by Jo Craven McGinty, Wall Street Journal, 7 September 2018
 
The article is subtitled: Indirect deaths—such as those caused by gaps in medication—can occur months after a storm, complicating tallies
   
Laura noted that
:[https://www.washingtonpost.com/news/fact-checker/wp/2018/06/02/did-4645-people-die-in-hurricane-maria-nope/?utm_term=.0a5e6e48bf11 Did 4,645 people die in Hurricane Maria? Nope.]<br>
:by Glenn Kessler, ''Washington Post'', 1 June 2018
 
The source of the 4645 figure is a [https://www.nejm.org/doi/full/10.1056/NEJMsa1803972 NEJM article].  Point estimate, the 95% confidence interval ran from 793 to 8498.
 
President Trump has asserted that the actual number is
[https://twitter.com/realDonaldTrump/status/1040217897703026689 6 to 18].
The ''Post'' article notes that Puerto Rican official had asked researchers at George Washington University to do an estimate of the death toll.  That work is not complete.
[https://prstudy.publichealth.gwu.edu/ George Washington University study]
 
:[https://fivethirtyeight.com/features/we-still-dont-know-how-many-people-died-because-of-katrina/?ex_cid=538twitter We sttill don’t know how many people died because of Katrina]<br>
:by Carl Bialik, FiveThirtyEight, 26 August 2015
 
----
[https://www.nytimes.com/2018/09/11/climate/hurricane-evacuation-path-forecasts.html These 3 Hurricane Misconceptions Can Be Dangerous. Scientists Want to Clear Them Up.]<br>
[https://journals.ametsoc.org/doi/abs/10.1175/BAMS-88-5-651 Misinterpretations of the “Cone of Uncertainty” in Florida during the 2004 Hurricane Season]<br>
[https://www.nhc.noaa.gov/aboutcone.shtml Definition of the NHC Track Forecast Cone]
----
[https://www.popsci.com/moderate-drinking-benefits-risks Remember when a glass of wine a day was good for you? Here's why that changed.]
''Popular Science'', 10 September 2018
----
[https://www.economist.com/united-states/2018/08/30/googling-the-news Googling the news]<br>
''Economist'', 1 September 2018
 
[https://www.cnbc.com/2018/09/17/google-tests-changes-to-its-search-algorithm-how-search-works.html We sat in on an internal Google meeting where they talked about changing the search algorithm — here's what we learned]
----
[http://www.wyso.org/post/stats-stories-reading-writing-and-risk-literacy Reading , Writing and Risk Literacy]
 
[http://www.riskliteracy.org/]
-----
[https://twitter.com/i/moments/1025000711539572737?cn=ZmxleGlibGVfcmVjc18y&refsrc=email Today is the deadliest day of the year for car wrecks in the U.S.]
 
==Some math doodles==
<math>P \left({A_1 \cup A_2}\right) = P\left({A_1}\right) + P\left({A_2}\right) -P \left({A_1 \cap A_2}\right)</math>
 
<math>P(E)  = {n \choose k} p^k (1-p)^{ n-k}</math>
 
<math>\hat{p}(H|H)</math>
 
<math>\hat{p}(H|HH)</math>
 
==Accidental insights==
 
My collective understanding of Power Laws would fit beneath the shallow end of the long tail. Curiosity, however, easily fills the fat end.  I long have been intrigued by the concept and the surprisingly common appearance of power laws in varied natural, social and organizational dynamics.  But, am I just seeing a statistical novelty or is there meaning and utility in Power Law relationships? Here’s a case in point.
 
While carrying a pair of 10 lb. hand weights one, by chance, slipped from my grasp and fell onto a piece of ceramic tile I had left on the carpeted floor. The fractured tile was inconsequential, meant for the trash.
<center>[[File:BrokenTile.jpg | 400px]]</center>
As I stared, slightly annoyed, at the mess, a favorite maxim of the Greek philosopher, Epictetus, came to mind: “On the occasion of every accident that befalls you, turn to yourself and ask what power you have to put it to use.”  Could this array of large and small polygons form a Power Law? With curiosity piqued, I collected all the fragments and measured the area of each piece.
 
<center>
{| class="wikitable"
|-
! Piece !! Sq. Inches !! % of Total
|-
| 1 || 43.25 || 31.9%
|-
| 2 || 35.25 ||26.0%
|-
|  3 || 23.25 || 17.2%
|-
| 4 || 14.10 || 10.4%
|-
| 5 || 7.10 || 5.2%
|-
| 6 || 4.70 || 3.5%
|-
| 7 || 3.60 || 2.7%
|-
| 8 || 3.03 || 2.2%
|-
| 9 || 0.66 || 0.5%
|-
| 10 || 0.61 || 0.5%
|}
</center>
<center>[[File:Montante_plot1.png | 500px]]</center>
The data and plot look like a Power Law distribution. The first plot is an exponential fit of percent total area. The second plot is same data on a log normal format. Clue: Ok, data fits a straight line.  I found myself again in the shallow end of the knowledge curve. Does the data reflect a Power Law or something else, and if it does what does it reflect?  What insights can I gain from this accident? Favorite maxims of Epictetus and Pasteur echoed in my head:
“On the occasion of every accident that befalls you, remember to turn to yourself and inquire what power you have to turn it to use” and “Chance favors only the prepared mind.”
 
<center>[[File:Montante_plot2.png | 500px]]</center>
   
My “prepared” mind searched for answers, leading me down varied learning paths. Tapping the power of networks, I dropped a note to Chance News editor Bill Peterson. His quick web search surfaced a story from ''Nature News'' on research by Hans Herrmann, et. al. [http://www.nature.com/news/2004/040227/full/news040223-11.html Shattered eggs reveal secrets of explosions].  As described there, researchers have found power-law relationships for the fragments produced by shattering a pane of glass or breaking a solid object, such as a stone. Seems there is a science underpinning how things break and explode; potentially useful in Forensic reconstructions.
Bill also provided a link to [http://cran.r-project.org/web/packages/poweRlaw/vignettes/poweRlaw.pdf a vignette from CRAN] describing a maximum likelihood procedure for fitting a Power Law relationship. I am now learning my way through that.
 
Submitted by William Montante
 
----

Latest revision as of 20:58, 17 July 2019


Forsooth

Quotations

“We know that people tend to overestimate the frequency of well-publicized, spectacular events compared with more commonplace ones; this is a well-understood phenomenon in the literature of risk assessment and leads to the truism that when statistics plays folklore, folklore always wins in a rout.”

-- Donald Kennedy (former president of Stanford University), Academic Duty, Harvard University Press, 1997, p.17

"Using scientific language and measurement doesn’t prevent a researcher from conducting flawed experiments and drawing wrong conclusions — especially when they confirm preconceptions."

-- Blaise Agüera y Arcas, Margaret Mitchell and Alexander Todoorov, quoted in: The racist history behind facial recognition, New York Times, 10 July 2019

In progress

What if the Placebo Effect Isn’t a Trick?
by Gary Greenberg, New York Times Magazine, 7 November 2018

The Problems With Risk Assessment Tools
by Chelsea Barabas, Karthik Dinakar and Colin Doyle, New York Times, 17 July 2019

Hurricane Maria deaths

Laura Kapitula sent the following to the Isolated Statisticians e-mail list:

[Why counting casualties after a hurricane is so hard]
by Jo Craven McGinty, Wall Street Journal, 7 September 2018

The article is subtitled: Indirect deaths—such as those caused by gaps in medication—can occur months after a storm, complicating tallies

Laura noted that

Did 4,645 people die in Hurricane Maria? Nope.
by Glenn Kessler, Washington Post, 1 June 2018

The source of the 4645 figure is a NEJM article. Point estimate, the 95% confidence interval ran from 793 to 8498.

President Trump has asserted that the actual number is 6 to 18. The Post article notes that Puerto Rican official had asked researchers at George Washington University to do an estimate of the death toll. That work is not complete. George Washington University study

We sttill don’t know how many people died because of Katrina
by Carl Bialik, FiveThirtyEight, 26 August 2015

These 3 Hurricane Misconceptions Can Be Dangerous. Scientists Want to Clear Them Up.
Misinterpretations of the “Cone of Uncertainty” in Florida during the 2004 Hurricane Season
Definition of the NHC Track Forecast Cone

Remember when a glass of wine a day was good for you? Here's why that changed. Popular Science, 10 September 2018

Googling the news
Economist, 1 September 2018

We sat in on an internal Google meeting where they talked about changing the search algorithm — here's what we learned

Reading , Writing and Risk Literacy

[1]

Today is the deadliest day of the year for car wrecks in the U.S.

Some math doodles

<math>P \left({A_1 \cup A_2}\right) = P\left({A_1}\right) + P\left({A_2}\right) -P \left({A_1 \cap A_2}\right)</math>

<math>P(E) = {n \choose k} p^k (1-p)^{ n-k}</math>

<math>\hat{p}(H|H)</math>

<math>\hat{p}(H|HH)</math>

Accidental insights

My collective understanding of Power Laws would fit beneath the shallow end of the long tail. Curiosity, however, easily fills the fat end. I long have been intrigued by the concept and the surprisingly common appearance of power laws in varied natural, social and organizational dynamics. But, am I just seeing a statistical novelty or is there meaning and utility in Power Law relationships? Here’s a case in point.

While carrying a pair of 10 lb. hand weights one, by chance, slipped from my grasp and fell onto a piece of ceramic tile I had left on the carpeted floor. The fractured tile was inconsequential, meant for the trash.

BrokenTile.jpg

As I stared, slightly annoyed, at the mess, a favorite maxim of the Greek philosopher, Epictetus, came to mind: “On the occasion of every accident that befalls you, turn to yourself and ask what power you have to put it to use.” Could this array of large and small polygons form a Power Law? With curiosity piqued, I collected all the fragments and measured the area of each piece.

Piece Sq. Inches % of Total
1 43.25 31.9%
2 35.25 26.0%
3 23.25 17.2%
4 14.10 10.4%
5 7.10 5.2%
6 4.70 3.5%
7 3.60 2.7%
8 3.03 2.2%
9 0.66 0.5%
10 0.61 0.5%
Montante plot1.png

The data and plot look like a Power Law distribution. The first plot is an exponential fit of percent total area. The second plot is same data on a log normal format. Clue: Ok, data fits a straight line. I found myself again in the shallow end of the knowledge curve. Does the data reflect a Power Law or something else, and if it does what does it reflect? What insights can I gain from this accident? Favorite maxims of Epictetus and Pasteur echoed in my head: “On the occasion of every accident that befalls you, remember to turn to yourself and inquire what power you have to turn it to use” and “Chance favors only the prepared mind.”

Montante plot2.png

My “prepared” mind searched for answers, leading me down varied learning paths. Tapping the power of networks, I dropped a note to Chance News editor Bill Peterson. His quick web search surfaced a story from Nature News on research by Hans Herrmann, et. al. Shattered eggs reveal secrets of explosions. As described there, researchers have found power-law relationships for the fragments produced by shattering a pane of glass or breaking a solid object, such as a stone. Seems there is a science underpinning how things break and explode; potentially useful in Forensic reconstructions. Bill also provided a link to a vignette from CRAN describing a maximum likelihood procedure for fitting a Power Law relationship. I am now learning my way through that.

Submitted by William Montante

Retrieved from "https://www.causeweb.org/wiki/chance/index.php?title=Sandbox&oldid=22455"