Best Books About Prediction

Prediction markets are often hailed as the gold standard of forecasting. These markets harness the "wisdom of crowds" by having people bet on the likelihood of future events. But teams of superforecasters did even better. Across a two year forecasting tournament, superforecaster teams bested prediction markets by 15-30%.

Why did teams do so well? The ability to share information and perspectives was key. Forecasters could share ideas, challenge each other, and collectively dig deeper into problems.

Diversity also played a big role. Teams with a variety of backgrounds and thinking styles generated more creative solutions. As long as discussions remained friendly and focused, diversity led to better accuracy.

Section: 1, Chapter: 8

Steve Ballmer's infamous 2007 forecast that "There's no chance that the iPhone is going to get any significant market share" looks hugely wrong in hindsight. But Ballmer never specified what "significant" market share meant, or what time period he was referring to. His forecast was too vague to definitely judge as right or wrong.

This is extremely common - and makes it effectively impossible to assess forecast accuracy. To be testable, forecasts need:

1. Specific definitions. What counts as a "default" or a "bubble" or a "coup"?
2. Precise time horizons. By what date will the event happen or not?
3. Numerical probabilities that can be scored. "60% chance" can be graded later as right or wrong; "pretty likely" cannot.
4. Repeated forecasts over time. One forecast is not enough - we need a track record.

Most real-world forecasts fail these criteria. As a result, we have little idea how accurate experts actually are, despite how much influence their predictions have.

Section: 1, Chapter: 3

The "tip-of-your-nose" perspective is how we intuitively perceive the world. It refers to both

1. the subjective vantage point we each have on reality, and
2. the tendency to treat our personal, close-up view as the truth, even when it's distorted or missing key facts.

For example, after 9/11, many Americans felt intensely anxious about terrorism and assumed more major attacks were imminent and inevitable. The tip-of-your-nose view made it feel that way. But taking an "outside view" by comparing the 9/11 death toll to other risks like heart disease, Americans' risk of dying in a terror attack was so low it was hardly worth worrying about.

Superforecasters know the tip-of-your-nose view is frequently misleading. It may "feel right" that a company is doomed to fail or that a war is unwinnable. But feelings are not a reliable guide to reality. Only by stepping outside ourselves and stress-testing our views against data can we avoid being misled.

Section: 1, Chapter: 5

One of the hardest things for any forecaster to do is to admit they were wrong. Humans are naturally resistant to acknowledging mistakes, due to cognitive dissonance and the pain of admitting error. We go to great lengths to rationalize failed predictions.

But superforecasters do the opposite. They are eager to acknowledge their misfires and examine why they happened. Some key practices:

• Meticulously tracking predictions so it's unambiguous when they fail
• Conducting "postmortems" to analyze the causes of mistakes
• Sharing lessons from failed forecasts with teammates to elevate the whole group
• Celebrating failed forecasts as learning opportunities, not shameful errors
• Revising their beliefs in light of results, even when it's uncomfortable

Superforecasters know there is no shame in being wrong. The only shame is in failing to acknowledge it or learn from it. By embracing their mistakes, they continuously sharpen their foresight.

Section: 1, Chapter: 7

Who are the superforecasters? They are a diverse group - engineers, lawyers, artists, scientists, Wall Streeters, and more. Many have graduate degrees, but some don't. They include a filmmaker, a mathematician, a pharmacist, and a retiree "looking to keep his mind active."

What they have in common is not so much who they are, but how they think. Superforecasters score highly on measures of fluid intelligence and actively open-minded thinking. They are numerate and capable of rapidly synthesizing information. But more important than raw intelligence is their cognitive style - they are actively open-minded, intellectually humble, eager to learn from their mistakes.

The superforecasters show that foresight isn't an innate gift, but a product of a certain way of thinking. And that way of thinking can be taught and cultivated - it doesn't require an elite background or PhD. It's an accessible skill.

Section: 1, Chapter: 4

As powerful as superforecasters are today, the future may belong to supersmart algorithms. Human Gut may soon meet Artificial Intuition as silicon superpredictors absorb the combined wisdom of carbon-based superforecasters.

IBM's Watson, for instance, can comb through millions of medical records to predict disease progression far faster and more accurately than doctors. Similar systems could soon be forecasting currency fluctuations, climate change impacts, and election results.

Still, humans will likely remain essential - not as solo forecasters, but as partners for AI. The key will be focusing human insight on what machines can't do well: Probing assumptions, generating novel scenarios, and making meaning from raw data. The result may be an "augmented intelligence" greater than either alone.

Section: 1, Chapter: 11

Many superforecasters have backgrounds in STEM fields and are highly numerate. They are comfortable with concepts like Bayes' theorem for updating probabilities based on new information. Some even build their own computer models.

But advanced math is not essential. Most superforecasters say they rarely use quantitative models or crunch numbers. Instead, their forecasting mainly relies on thorough research, careful reasoning, and sound judgment.

For Lionel Levine, a math professor, not using quantitative methods is a point of pride. He wants to prove he can be a great forecaster without relying on his mathematical toolkit: "It's all, you know, balancing, finding relevant information and deciding how relevant is this really? How much should it really affect my forecast?"

The key skills of balancing inside vs outside views, synthesizing perspectives, granular distinctions, and continual updating are accessible to anyone.

Section: 1, Chapter: 5

Philip Tetlock considers himself an "optimistic skeptic" when it comes to forecasting. The skeptical side recognizes the huge challenges of predicting the future in a complex, nonlinear world. Even small unpredictable events, like the self-immolation of a Tunisian fruit vendor, can have cascading consequences no one foresaw, like the Arab Spring uprisings.

However, the optimistic side believes foresight is possible, to some degree, in some circumstances. We make mundane forecasts constantly in everyday life. Sophisticated forecasts underpin things like insurance and inventory management. The key is to figure out what makes forecasts more or less accurate, by gathering many forecasts, measuring accuracy, and rigorously analyzing results. This is rarely done today - but it can be.

Section: 1, Chapter: 1

Despite the prevalence of forecasting in domains like economics, business, and politics, the track record of "experts" in these fields is dismal. For instance, economist Edgar Fiedler found that forecasts in a sample missed the mark by an average of 2.4 times the size of changes they were trying to predict. National intelligence agencies have failed to predict nearly every major geopolitical event in the past few decades. And Philip Tetlock's seminal study on expert political forecasts found that they barely outperformed random guesses. The deeper issue is that in complex systems like economies and societies, accurate prediction is essentially impossible due to the sheer volume of interacting variables and the potential for Black Swans.

Section: 2, Chapter: 10

Silver argues that many of our counterterrorism efforts, like overzealous airport security, amount to "security theater" that fails to address the biggest risks. We have a bias toward stopping familiar threats, even as terrorists employ new tactics.

The TSA confiscates cigarette lighters while a terrorist could simply blow up the security line. We fear Muslims from certain countries while homegrown extremists plot undetected.

Based on a statistical analysis, Silver estimates a 3% chance of a 100,000+ fatality terror attack per decade, most likely from nuclear or biological weapons. Yet policymakers often focus more on foiling numerous small-scale conventional plots that cause less total harm.

Our brains weren't wired for such low-probability/high-impact events, so we struggle with the correct response. But we must honestly weigh the probabilities, however uncomfortable, and allocate our limited resources accordingly.

Section: 1, Chapter: 13

he 2008 financial crisis represented a colossal failure of prediction by many of the institutions and individuals entrusted to forecast economic risk. Ratings agencies like Moody's and Standard & Poor's gave their highest AAA rating to mortgage-backed securities that were in reality extremely vulnerable to defaults. When the housing bubble burst, these securities failed at rates as high as 28%, compared to the 0.12% failure rate S&P had predicted for AAA-rated CDOs.

This predictive failure was widespread - from the ratings agencies to the banks issuing the securities to the regulators and economists who failed to sound adequate warnings. Incentive structures were poorly aligned, with entities like S&P being paid by the issuers of the securities they were rating. There was also a collective failure of imagination - an inability to consider that housing prices could decline significantly on a national basis. As a result, risks were severely underestimated, leading to the near-collapse of the global financial system when the housing bubble finally burst.

Section: 1, Chapter: 1

Weather forecasting is a field where the combination of human judgment and computer modeling has led to dramatic improvements in predictive accuracy. Today, thanks to advances in computing power, data collection, and modeling techniques, weather forecasts are far more accurate than they were even 20 or 30 years ago.

For example, the average error in a hurricane forecast track has been reduced from 350 miles (for a 3-day forecast) in 1984, to just 100 miles today. Temperature and precipitation forecasts have also become much more reliable. This improved accuracy has had major benefits, giving people more time to prepare for serious storms and saving countless lives.

Weather forecasting will never be perfect due to the inherently chaotic nature of the atmosphere. But the field demonstrates that substantial progress is possible with the right combination of scientific understanding, computational firepower, and human expertise. It's a model that other disciplines can learn from.

Section: 1, Chapter: 4

Computers and human minds have complementary strengths in forecasting. Computers have vast data-crunching power, perform complex mathematical simulations, and tirelessly consider every possibility. However, they lack contextual understanding and can only operate based on programming from humans. Skilled human forecasters supply the intuition, hypotheses, and insights that give a model's raw output meaning and utility in the real world.

The ideal approach combines the two, using human creativity to devise strategies and programs while leveraging computational power to do the grunt work of running the numbers. An example is how weather forecasts improved dramatically once meteorologists supplemented physical models of atmospheric dynamics with computer simulations.

Section: 1, Chapter: 9

One of the clearest lessons from economic history is to be deeply skeptical of overconfident economic forecasts, especially those that proclaim a "new era" of uninterrupted growth or that project present trends indefinitely into the future. Economists have a dismal record of predicting recessions and major turning points in the business cycle.

In the 2007-2008 financial crisis, for example, the median forecast from leading economists was that the economy would avoid recession and continue to grow. Even once the recession had already begun in December 2007, most economists still thought a recession was unlikely.

Part of the problem is incentives - bearish forecasts are often punished by markets and by clients who don't want to believe the party will ever end. There are also psychological biases at play, like recency bias (putting too much weight on recent events and performance) and overconfidence. Any projection that doesn't grapple with uncertainty and discuss the many ways the forecaster could be wrong is not worth very much.

Section: 1, Chapter: 2

In the early 1970s, the U.S. government launched a mass vaccination program against the swine flu, fearing a pandemic on the scale of the 1918 Spanish flu. However, the pandemic never materialized and the vaccines caused side effects, leading to a public backlash. Silver argues this failure stemmed from health officials making predictions from limited data points without considering the full context. Role models teach the importance of learning from history's mistakes and successes to inform our predictions about the future.

Section: 1, Chapter: 7

More information alone does not automatically lead to better predictions. In many fields, the growth of available data has outpaced our understanding of how to process it effectively. More data means more potential for spurious correlations, false positives, and noise obscuring the signal.

For example, the U.S. government now tracks over 45,000 economic statistics, exponentially more than even a few decades ago. But the number of genuinely causal and meaningful relationships in that data is orders of magnitude smaller. Most of it ends up being irrelevant noise when it comes to economic forecasting.

The challenge of the modern era is separating the valuable signal from the cacophony of noisy data. This requires focusing our predictions on areas where the data is most reliable and we have strong causal understanding. It also necessitates filtering the data to find the most relevant indicators.

Section: 1, Chapter: 8

In contrast to weather forecasting, the science of predicting earthquakes is still in its infancy. The history of the field is littered with false alarms, missed warnings, and overconfident but baseless predictions.

Seismologists have looked for predictive patterns in everything from animal behavior to electromagnetic signals to the timing of foreshocks. But none of these methods have delivered reliable predictions. The fundamental problem is that earthquake dynamics are extremely complex, nonlinear, and difficult to model. Current quake models are like "black boxes" - they can fit past data but have little predictive power.

Some argue that earthquakes are inherently unpredictable - a chaotic system where small changes in initial conditions can lead to vastly different outcomes. The jury is still out, but the track record to date suggests that reliable earthquake prediction is an extremely difficult challenge that will not be solved anytime soon.

Section: 1, Chapter: 5

Many people interpreted the book and movie Moneyball to mean that statistics and quantitative analysis were a guaranteed path to success in baseball, while traditional subjective scouting was obsolete. But this is an oversimplification of the book's message.

In fact, the most successful MLB teams today employ a hybrid approach that synthesizes both scouting and statistical analysis. Even the famously data-driven Oakland A's have significantly increased their scouting budget under GM Billy Beane, recognizing the importance of data that can't be fully captured by stats.

The lesson of Moneyball is not that statistics are inherently superior to scouting or vice versa. It's that the best forecasts come from a thoughtful synthesis of both subjective and objective information. The key is having an open mind, considering multiple perspectives, and not being wedded to any one ideology. This applies far beyond baseball.

Section: 1, Chapter: 3