Chance News 7

Sept 26 2005 to Oct 15 2005

quotation

While writing my book [Stochastic Processes] I had an argument with Feller [Introuction to Probability Theory and its Applications]. He asserted that everyone said "random variable" and I asserted that everyone said "chance variable." We obviously had to use the same name in our books, so we decided the issue by a stochastic procedure. That is, we tossed for it and he won.

Joe Doob
Statistical Science

Forsooth

Peter Winkler suggested our first forsooth.

Texas beats Ohio State in their opening game

of the season (Saturday Sept 10 2002). The sportscasters (legendary Brent Musburger on play-by-play or Gary Danielson on analysis) observed that of the 14 teams who have previously played in the championship game (at the end of each season) 5 have suffered an earlier defeat. "Thus," they conclude, "Ohio State can still make it to the championship game, but their chances are now less

than 50%."

Discussion

What is wrong with this?

---

Here are forsooths from a recent issues of RSS NEWS

'Big ticket quiz' at the start of Wimbledon:

Q. How many punnets (a small light basket or other container for fruit or vegetables) of strawberries are eaten each day during the Wimbledone tournament?

Is it (a) over 8,000, (b) over 9,000 or (c) over 10,000?

BBC radio 5
20 June 2005

Waiting time for foot surgery down by 500%

Evening News (Edinburgh)
5 July 2005

In 1996-8 when the number attending university was static, the participation of women was also static, but male participation fell.

The Times Higher
21 January 2005

[On the subject of congestion on the London Underground..] 'Last year 976 million of us used the tube...'

BBC London News
19 May 2005

Fortune's Formula

Fortune's Formula: Wanna Bet?
New York Times Book Section, September 25, 2005
David Pogue

This must be the kind of review that every Science writer dreams of. Pogue ends his review with:

Fortune's Formula may be the world's first history book, gambling primer, mathematics text, economics manual, personal finance guide and joke book in a single volume. Poundstone comes across like the best college professor you ever had, someone who can turn almost any technical topic into an entertaining and zesty lecture. But every now and then, you can't help wishing there were some teaching assistants on hand to help.

The author William Poundstone is a science writer who has written a number of very successful science books. His book, Prisoner's dilemma: John von Neumann, game theory and the puzzle of the bomb, was written in the style of this book. Indeed Helen Joyce, in her review of this book in Plus Magazine writes:

This book is a curious mixture of biography, history and mathematics, all neatly packaged into an entertaining and enlightening read.

Poundstone describes himself as a visual artist who does books as a "day job.". You can learn about his art work here.

Fortune's Formula is primarily the story of Edward Thorpe, Claude Shannon, and John Kelly and their attempt to use mathematics to make money gambling in casinos and on the stock market. None of these did their graduate work in mathematics. Thorpe and Kelly got their Phd's in physics and Shannon in Genetics.

In the spring of 1955, while a graduate student at UCLA, Thorpe joined a discussion on the possiblity of making money from roulette. Thorpe suggested that they could taking advantage of the fact that bets are still accepted for a few second after the croupier releases the ball, and in these seconds, he could estimate what part of the wheel the ball would stop.

Thorpe did not pursue this and in 1959 became an instructor in mathematics at M.I.T. Here he became interested in blackjack and developed his famous card counting method for wining at blackjack. He decided to publish his method in the most prestigious journal he could find and settled on Proceedings of the National Academy of Sciences. For this he needed to have a member of the Academy submit his paper. The only member in the math department was Shannon so he had to persuade him of the importance of his paper. Shannon not only agreed but in the process became fascinated by Thorpe's idea for beating roulette. He agreed to help Thorpe carry this out. They built a roulette machine in Shannon's basement. It worked fine there in trial runs but not so well in the casino so they did not pursue this method of getting rich.

Of course Thorpe is best known for showing that blackjack is a favorable game and giving a method to exploit this fact. Shannon is best known for his work in information theory. Kelly is known his method for gambling in a favorable game. This plays a central role in Poundstone's book and is probably why Pogue felt that it would help if he had a teaching assistant. Poundstone tries to explain Kelly's work in many different ways but what he really needed to understand it is an example but this required too many formulas for a popular book. So we shall include an example from Chance News 7.09.

Writing for Motley Fool, 3 April 1997 Mark Brady complained about the inumeracy of the general public and gave a number of examples including this one:

Fear of uncertainty and innumeracy are synergistic. Most people cannot do the odds. What is a better deal over a year? A 100% safe return with 5 percent interest or a 90 percent safe return with a 20 percent return. For the first deal, your return will be 5% percent. For the second, your return will be 8%. Say you invest $1000 10 times. Your interest for the 9 successful deals will be 9000 x 0.2 or 1800. Subtract the 1000 you lost on the 10th deal and you get a $800 return on your original $10,000 for 8 percent.

Peter Doyle suggested that a better investment strategy in this case is:

Faced with a 100 percent-safe investment returning 5 percent and a 90 percent-safe investment returning 20%, you should invest 20% of your funds in the risky investment and 80% in the safe investment. This gives you an effective return of roughly 5.31962607969968292 percent.

Peter is using a money management system due to J. L. Kelly (1956: "A new interpretation of information rate," Bell System Technical Journal, 35). Kelly was interested in finding a rational way to invest your money faced with one or more investment schemes, each of which has a positive expected gain. He did not think it reasonable to try simply to maximize your expected return. If you did this in the Motley Fools example as suggested by Mark Brady, you would choose the risky investment and might well lose all your money the first year. We will illustrate what Kelly did propose in terms of Motley Fools example.

We start with an initial capital, which we choose for convenience to be $1, and invest a fraction r of this money in the gamble and a fraction 1-r in the sure-thing. Then for the next year we use the same fractions to invest the money that resulted from the first year's investments and continue, using the same r each year. Assume, for example, that in the first and third years we win the gamble and in the second year we lose it. Then after 3 years our investment returns an amount f(r) where:

<math> f(r) = (1.2r + 1.05(1-r))(1.05(1-r))(1.2r + 1.05(1-r)). </math>

After n years, we would have n such factors, each corresponding to a win or a loss of our risky investment. Since the order of the terms does not matter our capital will be:

<math> f(r,n) = (1.2r + 1.05(1-r))^W(1.05(1-r))^L </math>

where W is the number of times we won the risky investment and L the number of times we lost it. Now Kelly calls the quantity:

<math> G(r) =\lim_{n \to \infty }\log(\frac{f( r,n)} {n}) </math>

the exponential rate of growth of our capital. In terms of G(r) our capital should grow like <math> e^{G(r)} </math>. In our example:

<math> \log(\frac{f(r,n)}{n}) = \frac{W}{n}\log(1.2r + 1.05(1-r)) + \frac{L}{n}\log(1.05(1-r)) </math>

Since we have a 90% chance of winning our gamble, the strong law of large numbers tells us that as n tends to infinity this will converge to

<math>

G(r) = 0.9\log(1.2r + 1.05(1-r))+0.1\log(1.05(1-r))

</math>

It is a simple calculus problem to show that G(r) is maximized by r = 0.2 with: G(0.2) = 0.05183. Then e^{0.05183} = 1.0532, showing that our maximum rate of growth is 5.32% as claimed by Peter.

The attractiveness of the Kelly investment scheme is that, in the long run, any other investment strategy (including those where you are allowed to change your investment proportions at different times) will do less well. "In the long run and less well" means more precisely that the ratio of your return under the Kelly scheme and your return under any other strategy will tend to infinity.

Thorpe had a very good experience in the stock market, which is described very well by Gaeton Lion in his excellent review of Fortunes's Formula. He writes:

Ed Thorpe succeeded in deriving superior returns in both gambling and investing. But, it was not so much because of Kelly's formula. He developed other tools to achieve superior returns. In gambling, Ed Thorpe succeeded at Black Jack by developing the card counting method. He just used intuitively Kelly's formula to increase his bets whenever the odds were in his favor. Later, he ran a hedge fund for 20 years until the late 80s and earned a rate of return of 14% handily beating the market's 8% during the period.

Also, his hedge fund hardly lost any value on black Monday in October 1987, when the market crashed by 22%. The volatility of his returns was far lower than the market. He did this by exploiting market inefficiencies using warrants, options, and convertible bonds. The Kelly formula was for him a risk management discipline and not a direct source of excess return.

Shannon also made a lot of money on the stock market, but did not use Kelly's formula. In his review Gaeton Lion writes

Claude Shannon amassed large wealth by recording one of the best investment records. His performance had little to do with Kelly's formula. Between 1966 and 1986, his record beat even Warren Buffet (28% to 27% respectively). Shannon strategy was similar to Buffet. Both their stock portfolios were concentrated, and held for the long term. Shannon achieved his record by holding mainly three stocks (Teledyne, Motorola, and HP). The difference between the two was that Shannon invested in technology because he understood it well, while Buffet did not.

Finally, Thorpe provided the following comments for the book cover:

From bookies to billionaires, you'll meet a motley cast of characters in this highly original, 'outside the box' look at gambling and investing. Read it for the stories alone, and you'll be surprised at how much else you can learn without even trying. --Edward O. Thorpe, author of Beat the Dealer and Beat the Market.

Which foods prevent cancer?

Which of these foods will stop cancer? (Not so fast)
New York Times, 27 September 2005, Sect. F, p. 1
Gina Kolata

Among other examples, the article includes a data graphic on purported benefits of dietary fiber in preventing colorectal cancer. Early observational studies indicated an association, but subsequent randomized experiments found no effect.

More to follow.

Slices of risk and the broken heart concept

How a Formula Ignited Market That Burned Some Big Investors, Mark Whitehouse, The Wall Street Journal, September 12, 2005.

This on-line article relates how a statistician, David Li, unknown outside a small coterie of finance theorists, helped change the world of investing.

The article focuses on a event last May when General Motors Corp's debt was downgraded to junk status, causing turmoil in some financial markets. The article gives a nice summary of the underlying financial instruments known as credit derivatives - investment contracts structured so their value depends on the behavior of some other thing or event - with exotic names like collateralized debt obligations and credit-default swaps.

The critical step is to estimate the likelihood that many of the companies in a pool of companies would go bust at once. For instance, if the companies were all in closely related industries, such as auto-parts suppliers, they might fall like dominoes after a catastrophic event. Such a pool would have a 'high default correlation'.

In 1997, nobody knew how to calculate default correlations with any precision. Mr. Li's solution drew inspiration from a concept in actuarial science known as the broken heart syndrome - people tend to die faster after the death of a beloved spouse. Some of his colleagues from academia were working on a way to predict this death correlation, something quite useful to companies that sell life insurance and joint annuities. He says:

Suddenly I thought that the problem I was trying to solve was exactly like the problem these guys were trying to solve,. Default is like the death of a company, so we should model this the same way we model human life."

This gave him the idea of using copulas, mathematical functions the colleagues had begun applying to actuarial science. Copulas help predict the likelihood of various events occurring when those events depend to some extent on one another. Until the events last May of this year, one of the most popular copulas for bond pools was the Gaussian copula, named after Carl Friedrich Gauss, a 19th-century German statistician.

Further reading

• The on-line article gives more details about what went wrong in the financial markets in May and the search for a more appropriate copula to capture better the broken heart syndrome between companies.
• Wikipedia is a very worthwhile on-line resource for definitions of technical words, such as copula.

Submitted by John Gavin.

Learning to speak via statistics and graph theory

Computer learns grammar by crunching sentences, Max de Lotbinière September 23, 2005, Guardian Weekly.
Profs’ New Software ‘Learns’ Languages, Ben Birnbaum, 9 Sep 2005, Cornell Daily Sun.

A language-learning robot may sound like science fiction but new software, developed by Cornell University psychology professor Shimon Edelman, with colleagues Zach Solan, David Horn and Eytan Ruppin from Tel Aviv University in Israel, is well on the way to constructing a computer program that can teach itself languages and make up its own sentences, the developers' claim.

Unlike previous attempts at developing computer algorithms for language learning - "Automatic Distillation of Structure," or "ADIOS" for short - discovers complex patterns in raw text by repeatedly aligning sentences and looking for overlapping parts. Once it has derived a language's rules of grammar, it can then produce sentences of its own, simply from blocks of text in that language.

It has been evaluated on artificial context-free grammars with thousands of rules, on natural languages as diverse as English and Chinese, on coding regions in DNA sequences and on protein data correlating sequence with function.

Edelman comments:

Adios relies on a statistical method for pattern extraction and on structured generalisations - the two processes that have been implicated in language acquisition. Our experiments show that Adios can acquire intricate structures from raw data including transcripts of parents' speech directed at two- or three-year-olds. This may eventually help researchers understand how children, who learn language in a similar item-by-item fashion, and with little supervision, eventually master the full complexity of their native tongue.

Plus Magazine's website offers a more logical explanation:

The ADIOS algorithm is based on statistical and algebraic methods performed on one of the most basic and versatile objects of mathematics - the graph. Given a text, the program loads it as a graph by representing each word by a node, or vertex, and each sentence by a sequence of nodes connected by lines. A string of words in the text is now represented by a path in the graph.

Next it performs a statistical analysis to see which paths, or strings of words, occur unusually often. It then decides that those that appear most frequently - called "significant patterns" - can safely be regarded as a single unit and replaces the set of vertices in each of these patterns by a single vertex, thus creating a new, generalised, graph.

Finally, the program looks for paths in the graph which just differ by one vertex. These stand for parts of sentences that just differ by one word (or compound of words) like "the cat is hungry" and "the dog is hungry". Performing another statistical test on the frequency of these paths, the program identifies classes of vertices, or words, that can be regarded as interchangeable, or equivalent. The sentence involved is legitimate no matter which of the words in the class - in our example "cat" or "dog" - you put in.

This last step is then repeated recursively.

The website uses graphs to illustrate some examples and it finishes with some reassuring words:

All this doesn't mean, of course, that the program actually "understands" what it's saying. It simply knows how to have a good go at piecing together fragments of sentences it has identified, in the hope that they are grammatically correct. So if, like me, you're prone to swearing at your computer, you can safely continue to do so: it won't answer back for a long while yet.

Further reading

The ADIOS homepage offers an overview and more detailed description of the program.

Submitted by John Gavin.</math>