Saturday 2014-11-29

Signal and the Noise by Nate Silver

Silver exhorts us to think in probabilities, and provides many public-policy-related paragraphs. DataAnalysisForPoliticsAndPolicy still provides a better grounding for this, though with more math.

Silver recounts how Voulgaris -- a professional gambler -- got started in gambling by betting his entire life savings of 80K. As the bet matured and the odds offered came back in line with his idea of reality, he could have hedged his position for a pure win. Since he had already bet all his savings, he didn't have the cash for the hedge, and could have lost the 80K, instead of locking in the win.

This is a bad process; Voulgaris should have bet the Kelly Criterion bet amount, which with his belief that the Lakers had a 25% chance, and with 6.5 to 1 odds offered, equals ( (6.5 * 0.25) - 0.75 ) / 6.5 = 13% of his life savings.

Finally, given that he was stuck with a mis-sized bet that had improved odds, he could have just gone to his bookie with the idea for the hedge, and his bookie would likely have done the deal for a fee.

Later in the book, Silver shows that he really is an excel geek, i.e. he explains Bayesian conditional probability via a spreadsheet. Here's a much simpler explanation:

+--------------------------------------------------------------------------------------------------+
|                                                                                                  |
|                                     PEOPLE                                                       |
|                                                                                                  |
+--------------------------------------------------------------------------------------------------+
  Of which, 80% of people are mathophobes
+---------------------------------------------------------------------------+----------------------+
|                                                                           |                      |
|                  MATHOPHOBES                                              |    MATHLETES         |
|                                                                           |                      |
+---------------------------------------------------------------------------+----------------------+
  Of which, we can correctly identify mathletes 75% of the time 
+---------------+-----------------------------------------------------------+----------------+-----+
|               |                                                           |                |     |
| WHOOPS        |  MATHOPHOBES                                              |    MATHLETES   |OOPS |
|               |                                                           |                |     |
+---------------+-----------------------------------------------------------+--------+-------+-----+
   Odds of finding a mathlete in the group we think are mathletes = MATHLETES / (MATHLETES + WHOOPS)       

A long-term study by Philip E. Tetlock of the University of Pennsylvania found that when political scientists claimed that a political outcome had absolutely no chance of occurring, it nevertheless happened about 15 percent of the time.
-- pretty good summary
Standard & Poors told investors, for instance, that when it rated a particularly complex type of security known as a collateralized debt obligation (CDO) at AAA, there was only a 0.12 percent probabilityabout 1 chance in 850that it would fail to pay out over the next five years.6 This supposedly made it as safe as a AAA-rated corporate bond7 and safer than S&P now assumes U.S. Treasury bonds to be.8 The ratings agencies do not grade on a curve. In fact, around 28 percent of the AAA-rated CDOs defaulted, according to S&Ps internal figures.
This was the idea that I set out to work onand slowly, over the course of those long days at KPMG in 2002, PECOTA began to develop. It took the form of a giant, colorful Excel spreadsheetfortuitously so, since Excel was one of the main tools that I used in my day job at KPMG. (Every time one of my bosses walked by, they assumed I was diligently working on a highly elaborate model for one of our clients.
Pedroia has what John Sanders calls a major league memorywhich is to say a short one. He isnt troubled by a slump, because he is damned sure that hes playing the game the right way, and in the long run, thats what matters.
The public at large became more interested in weather forecasting after the Schoolhouse Blizzard of January 1888. On January 12 that year, initially a relatively warm day in the Great Plains, the temperature dropped almost 30 degrees in a matter of a few hours and a blinding snowstorm came.26 Hundreds of children, leaving school and caught unaware as the blizzard hit, died of hypothermia on their way home.
to have been more reliable than his competitors in recent years. In November 2007, a time when most economists still thought a recession of any kind to be unlikely, Hatzius penned a provocative memo entitled Leveraged Losses: Why Mortgage Defaults Matter.
PokerKingBlog.com has alleged that Guy Lalibert, the CEO of Cirque du Soleil, lost as much as $17 million in online poker games in 2008,22 where he sought to compete in the toughest high-stakes games against opponents like Dwan. Whatever the number, Lalibert is a billionaire who was playing the game for the intellectual challenge and to him this was almost nothing, the equivalent of the average American losing a few hundred bucks at blackjack.
Small-scale terrorism is treated more like crime than an existential threat. What Israel certainly does not tolerate is the potential for large-scale terrorism