Chance News 106

Forsooth

You really are old as you feel
"Some of the subjects 'aged physiologically not at all [while] at the other extreme there were folks aging two to three times as much.'"

in: The Week, July 24, 2015, page 17 (online at theweek.com for subscribers).

Submitted by Chris Andrews

More on the hot hand

In Chance News 105, the last item was titled Does selection bias explain the hot hand?. It described how in their July 6 article, Miller and Sanjurjo assert that to determine the probability of a heads following a heads in a fixed sequence, you may calculate the proportion of times a head is followed by a head for each possible sequence and then compute the average proportion, giving each sequence an equal weighting on the grounds that each possible sequence is equally likely to occur. I agree that each possible sequence is equally likely to occur. But I assert that it is illegitimate to weight each sequence equally because some sequences have more chances for a head to follow a second head than others.

Let us assume, as Miller and Sanjurjo do, that we are considering the 14 possible sequences of four flips containing at least one head in the first three flips. A head is followed by another head in only one of the six sequences (see below) that contain only one head that could be followed by another, making the probability of a head being followed by another 1/6 for this set of six sequences.

TTHT	Heads follows heads 0 time
THTT	Heads follows heads 0 times
HTTT	Heads follows heads 0 times
TTHH	Heads follows heads 1 time
THTH	Heads follows heads 0 times
HTTH	Heads follows heads 0 times

A head is followed by another head six out of 12 times in the six sequences (see below) that contain two heads that could be followed by another head, making the probability of a head being followed by another 6/12 = 1/2 for this set of six sequences.

THHT	Heads follows heads 1 time
HTHT	Heads follows heads 0 times
HHTT	Heads follows heads 1 time
THHH	Heads follows heads 2 times
HTHH	Heads follows heads 1 time
HHTH	Heads follows heads 1 time

A head is followed by another head five out of six times in the two sequences (see below) that contain three heads that could be followed by another head, making the probability of a head being followed by another 5/6 this set of two sequences.

HHHT	Heads follows heads 2 times
HHHH	Heads follows heads 3 times

An unweighted average of the 14 sequences gives

[(6 × 1/6) + (6 × 1/2) + (2 × 5/6)] / 14 = [17/3] / 14 = 0.405,

which is what Miller and Sanjurjo report. A weighted average of the 14 sequences gives

[(1)(6 × 1/6) + (2)(6 × 1/2) + (3)(2 × 5/6)] / [(1×6) + (2 × 6) + (3 × 2)]

= [1 + 6 + 5] / [6 + 12 + 6] = 12/24 = 0.50.

Using an unweighted average instead of a weighted average is the pattern of reasoning underlying the statistical artifact known as Simpson’s paradox. And as is the case with Simpson’s paradox, it leads to faulty conclusions about how the world works.

Submitted by Jeff Eiseman, University of Massachusetts

Predicting GOP debate participants

Ethan Brown posted this following link on the Isolated Statisticians list:

The first G.O.P. debate: Who’s in, who’s out and the role of chance

by Kevin Quealy and Amanda Cox , "Upshot" blog New York Times, 21 July 2015

Because of the large number of declared candidates (16 and growing at the time of the article), Fox News has limited participation in its August 6 debate to those who meet the following criterion

Must place in the top 10 of an average of the five most recent national polls, as recognized by FOX News leading up to August 4th at 5 PM/ET. Such polling must be conducted by major, nationally recognized organizations that use standard methodological techniques.

But of course, polls are subject to sampling error. The NYT article uses simulation to illustrate how this could affect participation. Supposing the latest polling averages represent the "correct" values, they simulate 5 additional polls (as Ethan noted, this is a bootstrapping approach). The results demonstrate that this can affect who's in and who's out, as well as the order on the stage of those who do make the cut.

The Washington Post maintains a State of the debate widget that updates the current top 10 based on the most recent polling results. For July 31 we see

Note that the top figure is not a bar chart; it represents the positions of the candidates on the stage.