Predicting the Unpredictable (or at least tyring)

Reading Between the Lines
Are you smarter than Nate Silver?
Are you smarter than Nate Silver? // Creative Commons
By
Friday, January 25, 2013

Like many on the political left, I spent the final days of the 2012 election compulsively checking Nate Silver’s blog, FiveThirtyEight, now hosted on the website of The New York Times. It was an obsession. A way to calm myself, reassure myself everything was going to be alright. And I wasn’t alone — one in three clicks on the Times’ website in the week before the election was to FiveThirtyEight.

And we were vindicated! Silver’s predictions turned out to call all 50 states accurately, as well as nearly every Senate election. (What the hell, Heidi Heitkamp?) Just as Barack Obama’s future was secured, Nate Silver’s celebrity was hoisted to an even higher plane of greatness. The wunderkind statistician who rose from the obscurity of the blogosphere by calling 49 out of 50 states correctly in 2008 proved to his critics that it was not a one-time fluke, but rather the ultimate triumph of cold, hard data.

Someone had to get this guy a book deal!

So Penguin Publishing did. Mere days after the conclusion of the 2008 election, Silver signed a book contract for a rumored advance of almost $1 million. According to Silver’s book, “I was approached by a number of publishers who wanted to capitalize on the success of books such as ‘Moneyball’ and ‘Freakonomics’ that told the story of nerds conquering the world.” Four years later, the result hit shelves: “The Signal and the Noise: Why So Many Predictions Fail — But Some Don’t.”

The title of “The Signal and the Noise” refers to the common problem for statisticians of distinguishing the relevant piece of data — the signal — from the distracting ephemera — the noise. In the book, Silver seeks to answer why some predictions are better than others, as well as how we can improve predictions as a whole. To do this, Silver examines a vast array of prediction-driven issues, including the Great Recession, baseball scouting, political punditry, global warming, meteorology, chess, poker, earthquake science, the stock market, the spread of infectious diseases, terrorism and financial bubbles.

Silver’s thesis appears to center on a fundamental conflict in statistics — the tension between Bayesian and Frequentist analyses. I still don’t entirely understand the distinction, but it appears from Silver’s book that the main contrast is over the definition of probability itself: Frequentists believe that probability is objective, while Bayesians believe that probability is subjective. Silver comes down firmly on the side of the Bayesians.

“The Signal and the Noise” is at its most interesting when Silver discusses topics with which he has personal experience. When discussing the statistics involved in online poker, Silver reminisces about his own days as a reasonably successful professional poker player. When discussing baseball, Silver dwells on his early success in mathematizing baseball scouting. When discussing politics, Silver gives us a little of the backstory behind the FiveThirtyEight blog.

But it seems to me that there is far too little context given — ironic, as Silver’s entire thesis is about the importance of “the underlying context.” Silver tells us fascinating tidbits about meteorology (it’s far more accurate than we believe) and earthquakes (we really can’t predict them at all) and chess (world champion Garry Kasparov only lost to the computer Deep Blue because he essentially psyched himself out), but he doesn’t tell us too much about his own life or FiveThirtyEight. As an avid reader of FiveThirtyEight, I bought the book for that reason!

Furthermore, “The Signal and the Noise” is not the easiest read. While the topics it discusses are fascinating, I feel that you sometimes need a higher understanding of statistics to really get a lot out of the book. Take this sentence, which came out of nowhere: “Thus, economists debate whether consumer confidence is a leading or lagging indicator, and the answer may be contingent on the point in the business cycle the economy finds itself at.” Without really defining any of the terms, and then immediately moving on, Silver left me a little lost.

“The Signal and the Noise” is an excellent attempt to teach the reader how to judge predictions. Most of the book, reviewers have noted, are about issues that modern statistics cannot predict — in the stock market, in predicting climate change, in the housing bubble, in predicting natural disasters. Perhaps this is why “The Signal and the Noise” is somewhat unsatisfying to me. In spite of Silver’s decent attempt, the book remains largely inaccessible to me. Furthermore, it’s not all that relevant. Sure, now I know not to trust anyone yelling about an earthquake, but I don’t know that much more about Nate Silver.

Comments