The Signal and the Noise

Starred review from December 24, 2012
Despite the fact that there is more information about everything from finance to professional sports available than ever before, predictions "may be more prone to failure" in this "era of Big Data." Balancing technical detail and thoughtful analysis with fluid prose, statistician Silver (FiveThirtyEight ) picks apart the many ways in which predictions in various fields have been flawed, while suggesting approaches that could improve the practice. The catastrophic miscalculations on the part of financial lending agencies that led to the recession of 2008 arose for the same types of reasons that caused baseball scouts to undervalue Boston Red Sox all-star player Dustin Pedroia or feed into a political pundit's flawed forecast: overconfidence in models based on oversimplified principles and unrealistic initial assumptions. Though there is no simple solution, a Bayesian methodology, in which prior beliefs are taken into account and initial assumptions constantly revised, would lead to more accurate predictive models. Effective prediction requires, according to Silver, "the serenity to accept the things we cannot predict, the courage to predict the things we can, and the wisdom to know the difference." Agent: Sydelle Kramer, The Susan Rabiner Literary Agency.