Is Poker predominantly a game of skill or luck?

Harvard ponders just what it takes to excel at poker.
Wall Street Journal, May 3, 2007, A1.
Neil King JR.

The WST article reports on a one-day meeting in the Harvard Faculty Club of poker pros, game theorists, statisticians, law students and gambling lobbyists to develop a strategy to show that poker is not predominantly a game of chance.

In the article we read:

The skill debate has been a preoccupation in poker circles since September (2006), when Congress barred the use of credit cards for online wagers. Horse racing and stock trading were exempt, but otherwise the new law hit any game "predominantly subject to chance". Included among such games was poker, which is increasingly played on Internet sites hosting players from all over the world.

This, of course, is not a new issue. For example it is the subject of the Mark Twain's short story "Science vs. Luck" published in the October 1870 issue of The Galaxy. The Galaxy no longer exists but co-founder Francis Church will always be remembered for his reply to Virginia's letter to the New York Sun, "Yes, Virginia, there is a Santa Claus".

In Mark Twain's story a number of boys were arrested for playing "old sledge" for money. Old sledge was a popular card game in those times and often played for money. In the trial the judge finds that half the experts say that old sledge is a game of science and half that it is a game of skill. The lawyer for the boys suggests:

Impanel a jury of six of each, Luck versus Science -- give them candles and a couple of decks of cards, send them into the jury room, and just abide by the result!

The Judge agrees to do this and so four deacons and the two dominies (Clergymen) were sworn in as the "chance" jurymen, and six inveterate o.ld seven-up professors were chosen to represent the "science" side of the issue. They retired to the jury room. When they came out, the professors had ended up with all the money. So the Judge ruled that the boys were innocent.

Today more sophisticated ways to determine if a gambling game is predominantly skill or luck are being studied. Ryne Sherman has written two articles on this, "Towards a Skill Ratio" and "More on Skill and Individual Differences" in which he proposes a way to estimate luck and skill in poker and other games.

To estimate skill and luck percentages Sherman uses a statistical procedure called analysis of variance (ANOVA). There many discussion of ANOVA on the web. One of these Variance and the Design of Experimentsl begins with the following hypothetical data.

 

Treatment 1	Treatment 2
4	7
6	5
8	8
4	9
5	7
3	9

These might be the result of a clinical trial to determine if vitamin WR improves your memory. In the study one group is given a placebo and the second group takes vitamin WR for a month. At the end of the month the two groups are given a memory test and the numbers in the columns represent the number of correct answers the participants had. Then an ANOVA test is made to see if there is a signicant difference between the groups.

Here is Bill Peterson's explanation for how this works.

There are two group means:

mean1 = (4+6+8+4+5+3)/6 = 30/6 = 5.0
mean2 = (7+5+8+9+7+9)/6 = 45/6 = 7.5

Then a grand mean over all observations
Mean = (30+45)/(6+6) = 6.25

Variance is always a sum of square deviations divided by degree of freedom: SS/df. This is also called a mean squared deviation MS.

The idea of ANOVA is to compare variation between groups (which measures the treatment effect) to the variation within groups (which is noise or error). To this end, we express the deviation from the grand mean as a sum of two parts: the difference from the group mean ("within") and the difference of the group mean from the grand mean ("between").

Thus:
(4 - 6.25) = (4 - 5.0) + (5.0 - 6.25)
(6 - 6.25) = (6 - 5.0) + (5.0 - 6.25)
...
(3 - 6.25) = (3 - 5.0) + (5.0 - 6.25)

(7 - 6.25) = (7 - 7.5) + (7.5 - 6.25)

(5 - 6.25) = (5 - 7.5) + (7.5 - 6.25)
...
(9 - 6.25) = (9 - 7.5) + (7.5 - 6.25)

These are obvious arithmetic. The magic (actually the Pythagorean Theorem)
is that the sums of squares decompose in this way.

(4-6.25)^2 +...+(9-6.25)^2 =
[(4-5.0)^2+...+(9 - 7.5)] + [(5.0 - 6.25)^2+...+(7.5 - 6.25)^2]
Check: 46.25 = 27.5 + 18.75

In the usual abbreviations:

SST = SSE + SSG

(total sum of sqs = error sum of sqs + group sum of sqs)

Fisher's F statistic is F = MSG/MSE. Large values of F are effectively a large signal to noise ratio, and we conclude that there is a real treatment effect.

We shall show how defines his measure of skill and of chance for a game using the following data from five weeks of our low-key Monday night poker games.

	Sally	Laurie	John	Mary	Sarge	Dick	Glenn
Game 1	-6.75	-10.10	-5.75	10.35	9.7	4.43	-1.95
Game 2	4.35	-4.25	.40	-.35	-8.8	-.15	5.8
Game 3	6.95	-4.35	.18	-7.75	7.65	-5.9	3.9
Game 4	-1.23	-11.55	4.35	2.9	4.85	-3.9	3.25
Game 5	6.35	-1.5	-.45	-.65	-.25	-4.9	1.42

To compare the amount of skill and luck in these games Shermon would have us carry out an analysis of variance in the same way we did for our example. The variation between groups is the variation in the players winnins throught the games. Shermon believes that this variation is due primarily to luck. The within variation is the variation of the players winnings betwee the games and this variatioin he believes this variation is primarily due to skill.

The within group sums of squares estimates the variation of the winnings within players and Sherman assumes that this is primarily due to luck. This leads him to define the skill factor as the ratio of the between group sums of squares to the total sums of squares and the luck factor as the ratio of the within group sums of squares to the total sums of squares. This gives us a skill % and a luck %. Using our poker data and the SAS ANOVA program we find that the total sums of squares is 1069.95, the between groups the sums of squares is 311.447and the within groups the sums of squares is 758.499. Thus from our poker data we would estimate the skill % to be 311.447/1069.95 = 29.1% and the luck % to be 758.499/1069.06 = 70.9%. Thus not surprisingly for our games luck is more important than skill.

In his second article, Sherman reports the skill % he obtained using data from a number of different types of games. For example, using data for Major League Batting hits the skill% was 39% and for home runs it was 68%. For NBA Basketball for points scored it was 75% and for poker stars in weekly tournaments it was 35%.

Sherman concludes his articles with the remarks:

If two persons play the same game, why don't both achieve the same results? The purpose of last month's article and this article was to address this question. This article suggests that there are two answers to this question: Skill (or systematic variance) or Luck (or random variance). Using both the correlation approach described last month and the ANOVA approach described in this article, one can estimate the amount of skill involved in any game. Last, and maybe most importantly, Table 4 demonstrated that the skill estimated involved in playing poker (or at least tournament poker) are not very different from other sport outcomes which are widely accepted as skillful.<\blockquote>

Discussion questions

(1) Do you think that Sherman's measure of skill and luck in a game is reasonable? If not why not?

(2) There is a form of duplicate poker modeled after duplicate bridge. Do you think that the congressional decision should not apply to this form of gambling?

Submitted by Laurie Snell