The predictions of experts consume our news cycle. As society struggles with a pandemic, our first inclination is to find “experts” who have years of academic focus on a particular topic and then listen closely to their perspective. The desire to seek the counsel of “experts” makes sense when the scenario is outside the scope of our understanding – like a pandemic. We trust that years of study and research will arm these “experts” to make much more accurate predictions, thus informing our own decisions. However, recently we have seen “experts” struggle to make accurate predictions – See British Professor Fired for poor prediction, Arizona health official BEG governor to act – he doesn’t, Covid number still fall, experts predicted there would be NO WAY the NFL could have a successful season, in November 2020 the Institute for Health Metrics and Evaluation predicts by February 1 India to have thousands of deaths per day. Still, as of February 6, India has less than 100.
So what gives? Why are the smartest people on a specific topic so inconsistent at making predictions? In his 2015 book, Super Forecasters, Dr. Philip Tetlock explains that the average “expert” is as statistically accurate as a dart-throwing chimp. YIKES. The problem for experts making predictions is that the academic rigor, focus, and intelligence required to become an expert are the same traits that cause experts to hold fast to their beliefs. The stubborn nature of high-achieving experts leads to an inability to make accurate predictions. Dr. Tetlock found that “experts” rarely seek out new evidence that undercuts their first conclusions, and after years of commitment and study towards developing a “first conclusion,” this places “experts” in a difficult, if not impossible, position. Any new information threatens the tightly held, hard-fought, expensive belief that makes that person an expert. Tetlock noted that often “experts” replace the HARD question with the EASY one, making their connection to currently held beliefs almost impossible to release. (See Availability Heuristic) An interesting metaphor for this scenario is the Fox versus the Hedgehog. A Hedgehog knows one thing. It doesn’t matter the circumstance; the Hedgehog knows that curling up in a ball is an effective, smart tactic and has a high probability of success. But the Fox does not have such a luxury. The Fox must use a wide range of strategies to survive. He doesn’t have one thing, so he must make the best decision with the most recent information available. Experts are hedgehogs. Super Forecasters are foxes. When it comes to predictions, you want foxes, not hedgehogs.
Tetlock describes the philosophical view of super forecasters as cautious, humble, and non-deterministic. These super forecasters tend to be open-minded, intellectually curious, reflective, and comfortable with numbers. In “how” they forecast, these super forecasters are not attached to any specific idea or agenda; they can consider other points of view, value diverse perspectives, think in a probabilistic manner, and are thoughtful updaters (meaning they change their views when they obtain new facts).
Are these the adjectives you would use to describe the pontificating pundits on the news networks – either right-leaning or left-leaning? Do the “experts” you base your own opinions on possess these characteristics? In situations like pandemics, with millions of deaths, tens of millions unemployed, and out of school, shouldn’t we be more focused on getting it right, not being right?
As we all try to navigate well-intended predictions about Covid, here are some questions to consider when deciding whether or not to heed the predictions cast by your favorite “expert”:
1. If your favorite Covid expert has not updated their predictions and advice since the beginning of the pandemic, you might want to question their ability to make accurate predictions?
2. If your favorite Covid expert has been wrong in a way that is statistically significant, you might want to question their ability to predict.
These questions aren’t meant to cast doubt on a person’s expertise but are intended to illustrate the fact that being an expert does not automatically make you good at predicting future outcomes. The skills required to become an expert and make expert predictions are not the same, so it is up to all of us to be thoughtful updaters - and evolve our viewpoint around experts. In a time of crisis, we need to focus on following the most accurate predictions, which in the end is probably NOT coming from an expert.