Book review: Nate Silver, The Signal and the Noise Featured

Rate this item
(2 votes)

"The art and science of prediction"

Nate Silver is the man who predicted the result in every state in the 2012 US election, projecting him to instant stardom amongst political pundits.

This is a book about the tools of Silver's trade, rather than being explicitly about how he achieved that exceptional result. He offers tips and tricks on how to better interpret the world we live in and on making good predictions. He is modest about his achievement and cautious about the limits of prediction. It is well written, in an accessible style, mixing humour and anecdote with good research and methods, making what could have been a dry read quite entertaining. His destruction of television pundits is very good.

Silver ranges far and wide in order to give us an understanding of his field. There are chapters on the banking credit crisis, weather forecasting, earthquakes, poker, sports betting, politics, terrorism and climate change.  Even if you really care nothing for, say, baseball, you will learn useful things about the world we live in that can be generalised and applied to areas close to your heart.

Bayes Theorem

At the core of his book is the message that we cannot know everything. We can never pin down reality perfectly. We can only be "less and less wrong".

How can we be less and less wrong?  Silver gives us Bayes Theorem. We have to know something about the world, enough to be able to make probabilistic estimates. With that, we can become less wrong with each morsel of new information. How? We need three things to enter into Bayes Theorem:

1) Our 'Prior' - our estimate that an event might happen, made before any new information;
2) 'The Given' - our estimate that an event has happened/could happen, given something else impacting on the scenario (The Hypothesis);
3) A measure of 'Wrong' - our estimate of the probability that the event has/will happen in the absence of The Given occurring (the probability that The Hypothesis is false).

Even if you know this stuff, Silver writes well and provides useful context. If you have trouble grasping it, Silver gives quite a few examples of how it works in practice, from cheating spouses to planes hitting the World Trade Centre.  You can find further examples quite easily by googling, say, "bayes theorem real examples". As an aside, Bayes will also help to prevent you from succumbing to the Base Rate fallacy.


Frequentists versus Bayesians

Some readers may be aware of the Frequentist versus Bayesian debate. Silver puts frequentism into context, explaining why we need a frequentist approach, but also why it is exposed to inherent flaws. The flaws can be summed up as "because we are human" and because measuring tools and methods have (often hidden) inbuilt biases. The doyen of frequentism, R A Fisher, argued that cancer caused smoking, rather than the other way around.  I should make clear that Fisher was a very bright man, highly respected, and that frequentism is an important, useful tool.

Frequentism "emphasizes the objective purity of the experiment", supposing "every hypothesis could be tested to a perfect conclusion if only enough data were collected." In doing so, it denies the human interaction in the process.  The plausibility of our hypothesis is extremely difficult to question because it relies on our beliefs. After all, what else do we have? We are imprisoned with the bubble of our fallible belief system.  The casual reader may dismiss this statement with, "Well, I'm very good at distinguishing the implausible". And indeed, generally, we are very good at distinguishing the implausible.  However, we do so from within our existing belief systems and we are prone to fitting a plausible story to the available information. We are fallible.

Anyone familiar with the variety of psychological biases will understand the problem (see Kahneman, Thinking Fast  and Slow). The difficulty pollsters had in estimating the result of the recent Scottish referendum is a good example.  Added to the pollsters problem was that there were, apparently, fewer nationalists than polls showed there were voters prepared to vote for the nationalist cause, giving them an unfortunate explanatory gap (my explanation: protest votes).

False Positives

One of the most useful sections of the book arises from a discussion around false positives. Silver references Ionnidis' 2005 paper "Why Most Published Research Findings Are False". As Silver says, "the failure rate for predictions made in entire fields ranging from seismology to political science appears to be extremely high".  Too much science is based on false positive results - even in renowned research faculties and in unexpected fields (confirmed, rather, by Fisher above). The issue of false positives is this:

a) there is an awful lot of data;
b) therefore there are an awful lot of apparent causal relationships, chance patterns, in Big Data;
c) there are very few meaningful causal relationships in data;
d) therefore we are highly likely to find false relationships in the noise.

If we combine this with the human propensity to derive plausible explanations from available information then we have a potent cocktail for a hangover.

If you are interested in problems of epistemology and ontology (aren't we all?) then this provides some useful spin. How do we know that we are right? How do we know that we are not operating with a picture built from false positives? We don't. This is a tragedy of the human condition, we can never really 'know'.  We have to appreciate and learn to live with uncertainty, while aiming to be, at best, less wrong.

Silver provides us with an excellent overview and context. This is stuff you need to know if you want to improve your grasp on reality. I heartily recommend his book.

Postscript

I make a point of not reading other reviews until mine is written. A sound complaint in many other reviews is that the book suffers a little from readers' difficulty in picking out the signal from the noise in the book itself.  Yes, this is a fair charge. The section treating Bayes Theorem is quite small and buried deep in the middle. The book benefits from a rereading. This does not mean the other chapters are of little value. Each is used to introduce a great number of related ideas, from the difference between risk and uncertainty, to Philip Tetlock's foxes and hedgehogs.

Another complaint is that Silver's treatment of the wide range of topics is a little shallow and he gives away how little he knows in some areas. Given the breadth of material covered this is to be expected - the book would be huge if he went into greater depth, while Silver would be a quite phenomenal brain if he had in-depth expertise in all the areas covered.  I was relaxed about this as the principles were clear from the examples, but the reader should beware putting down the book thinking that they have a good insight into the credit crunch or the spread of infectious disease - the treatment is too brief (the book might have benefited from being seen by more of the relevant experts prior to publishing).

Those well versed in the practical applications of Bayes also complain that there is little maths and a lot more that could be said. This is true, but the book is aimed at more general readers and does a good job of bringing Bayes to their attention. If you already work in the field, there is no new methodology to discover here. That said, there is useful wider context and there are many 'experts' who might benefit from the hints that humans are fallible and overconfident in some of their assertions and uses of data.

The book left me thinking that I would love to see more data and analyses of rugby and cricket, instead of the volumes of questionable talking-head punditry we get. Rather than TV producers targeting the lowest common denominator, why don't they set the bar a little higher and drag the audience up?


© Priory Orchard Ltd

Last modified on Friday, 12 December 2014 11:06
Login to post comments