Monday, December 17, 2012

"The Signal and the Noise"

Nate Silver's first work of non-fiction reads more like an enthusiastic extended commentary within a science or economic journal, especially when compared with works by contemporaries like Malcolm Gladwell. This is no accident: Silver is a statistician, a numbers and predictions wonk. He's not into (nor, to my knowledge, extensively trained in) descriptive writing, and the brief interludes his accounts of discussions with scientists, politicians, and experts in various fields, his prose becomes awkward. But then he starts talking numbers. And the chapter sails on. Flush with new approaches to data analysis in a complicated world.

All that said, the analysis is stellar, and in the spirit of Freakonomics, S&N encourages us to approach real-world problems with a few great ideas:
1. A Bayesian approach is reasonable, especially when dealing with the probability of an event happening, and in which prior information ("Bayesian priors") is available.
2. "Noise" comes in several forms: data points that neither reflect trends nor are truly significant (especially true of rare events); noise is in the news all the time (storms, accidents, global warming, political polls, commonly held beliefs that are statistically un-proven and may in fact be completely wrong); trends in sports; misperceptions about money, the stock market, economies in general.
3. "Signal" is more of a pseudonym for well-collected and well-organized data that are neither 1) over-fit or 2) meaningless because of the uncertainty in of the prediction (in time or space); "Signal" also takes into account known knowns, known unknowns, and tries, as best it can, to leave wiggle room for the dreaded unknown unknowns.

Anyway, it's a good book. Give it a read!

No comments:

Post a Comment