The Downfall (and Possible Salvation) of Expertise

In 2013, Northwestern’s Paola Sapienza and Chicago Booth’s Luigi Zingales, the researchers behind the trust index, analyzed whether Americans trust and agree with economists. They posed policy and economics questions to both ordinary Americans as well as US economists on Booth’s Economic Experts Panel, run by the Initiative on Global Markets, which periodically surveys several dozen senior faculty—including Nobel laureates, John Bates Clark medalists, past members of the president’s Council of Economic Advisers, and other similarly recognized economists—at elite research institutions. The average American, whose opinion was gleaned from the quarterly Financial Trust Index surveys of representative samples of US citizens, tended to disagree with economic experts, particularly on topics where experts agreed strongly with one another. And nonexperts maintained their opinions even when they learned what the economists had to say.

Distrust in experts is not exclusive to the US. Chicago Booth’s Michael Weber tapped into the IGM European Economic Experts Panel with research partners in Italy and Germany, Luigi Guiso of the Einaudi Institute for Economics and Finance and Ifo Institute’s Sebastian Link. They asked nonexpert Europeans (as well as central-bank economists and German executives) in January about topics ranging from immigration to competition with China, artificial intelligence, and the merits of raising the retirement age. They then tested whether respondents’ opinions changed after learning what the experts on the panel thought.

Though the responses are still being analyzed, Weber notes that he was struck by two early results. First, while expert and lay opinion were more closely aligned in Europe than in the US, lay/nonexpert Europeans were similarly unlikely to be swayed by expert opinion. Second, the researchers find a pattern of polarization. After being told of consensus among experts, respondents with an interest in public policy or economics were more likely to change their opinions to match those of experts, but respondents without those interests, or who had previously indicated they distrusted academic expertise, moved in the opposite direction. “This is something you imagine in the US, but it’s there in Europe too,” says Weber, who grew up outside of Heidelberg, Germany, in a small town whose residents he describes as conservative and respectful of expert opinion.

During the COVID-19 pandemic, residents of that town have observed social-distancing rules with no discernible resistance, Weber says. But elsewhere in Germany, protests against lockdowns demonstrated that “there’s a fraction of the population there that is not interested in what scientists say is safe,” he laments.

Similar polarization extends beyond Europe, according to Edelman, which for two decades has published an annual index probing public trust in nongovernmental organizations, businesses, government, and media. The 2020 index numbers show the gap in trust levels between the informed public—college-educated high earners who report an interest in public policy and business news—and the remaining mass population has increased six percentage points since 2012. The credibility of academic experts follows a similar trend: in 2012, 65 percent of the mass population and 69 percent of informed public respondents said academics were very or extremely credible; since then, the proportion has flatlined in the mass population and climbed to 71 percent of the informed public.

Why the distrust?

According to Sapienza and Zingales, the lack of trust—in economists, at least—stems from the fact that experts and nonexperts often start with different unspoken assumptions, which leads them to different conclusions about issues such as the North American Free Trade Agreement. For example, in the researchers’ 2013 study, more experts than nonexperts considered government to be trustworthy. The experts, who trusted government to, say, neutralize the painful effects of trade agreements, said that NAFTA has benefited US citizens. Nonexperts, with less faith in government, didn’t share that conclusion. Similarly, the experts saw a decade of rising gas prices as the result of market forces, while nonexperts saw it as the result of energy policy.

Why don’t nonspecialists reason their way through verdicts as dispassionately as experts seem to? And if they can’t, why don’t they acknowledge their knowledge gaps and emotional diversions, and cede this ground to the experts? Tonia Ries, executive director of the Edelman arm that runs the Edelman Trust Barometer, an annual trust and credibility survey, says the general public finds experts more credible than CEOs, celebrities, government officials, and regulators—but survey respondents rank another category of person as nearly as credible as experts: “a person like yourself.”

But research by University of California at Berkeley’s Stefano DellaVigna and Chicago Booth’s Devin G. Pope suggests that laypeople have some reason to trust themselves in addition to experts; and yet, they are not especially good at recognizing how much faith to put in their own predictions. For example, they are bad at estimating how much their own experience and knowledge can or will inform smart decision-making.

DellaVigna and Pope asked experts and nonexperts to predict the results of an experiment related to incentivizing certain behavior.

Laypeople’s forecasts, especially when aggregated together, were often as good as those of the experts in the study. The average forecast among 208 professors was highly accurate and better than the average forecasts by nonexperts (graduate students, undergraduates, and participants on the crowdsourcing website Amazon Mechanical Turk, who were meant to represent laypeople). Yet by grouping together the forecasts of lay predictors, the researchers find that they were as good as those of the best individual experts.

These findings chime with work by University of Pennsylvania’s Philip Tetlock, whose research over 30 years has suggested certain amateurs are better at accurately predicting outcomes than narrowly focused experts are, even when those predictions are in the professionals’ fields of expertise. He compared these lay forecasters with the polymath “foxes,” a description by the late philosopher Isaiah Berlin for people who see the world through a range of prisms, as opposed to “hedgehogs,” who tend to interpret events through one main framework. For example, Norwegian playwright Henrik Ibsen, according to Berlin, was a hedgehog—his single organizing principle in his work being the need for individuals to break free from societal mores—whereas Shakespeare, a fox, played with a wide array of ideas, experiences, and philosophies that sometimes even contradicted one another.

Tetlock’s results have been embraced by at least one US policy-making institution: the Office of the Director of National Intelligence, which began a process about 10 years ago of identifying (and then honing the skills of) laypeople “superforecasters” whose predictions of possible future global events could be weighed alongside those of trained experts.

Pope cautions that some factual questions are still best asked of experts. “Don’t go to your neighbors if you think you have cancer,” he says, “go to a doctor for that.”

There are, however, certain types of advice, says Pope, in which predictions must be made despite considerable uncertainty, and these might be enriched by a wider scope of experience. For example, when students ask him for help finding a thesis or dissertation topic, he urges them to consult other professors, too, to make use of the wisdom of crowds; the averaged predictive powers of experts in his study with DellaVigna were very good, after all. But Pope suggests his students also talk to their fellow students: “As long as you keep scaling up the lay expertise, it can be as effective as or even more so than the experts.”

Experts for tough choices

There are more reasons to trust experts than just their odds of getting it right—time saving, for example. Karina de Kruiff, a Chicago native who moved to Munich in 2013, is not sure she agrees with all the guidelines the government has issued about COVID-19, but she has largely followed them, returning to work as a kindergarten teacher in late April despite some nervousness about the number of children she would be caring for, and heading to the Alps with her women’s hiking group in May despite it feeling “like we were doing something wrong the whole time.” Whereas she sees friends in the US questioning everything, “here, people do their research but they also say, ‘OK, if it’s allowed, I’ll do it.’”

Research by Samantha Kassirer at Northwestern and Chicago Booth’s Emma Levine and Celia Gaertig suggests this latter approach—to trust experts with tough choices—emerges most often in situations that have a high degree of uncertainty. In a recent study, the researchers investigated how participants reacted to different types of medical advice. Some advice was “paternalistic,” in which hypothetical doctors recommended a specific course of action on the basis of their personal opinion, while other advice was more focused on patient autonomy, and the doctors resisted suggesting a path. The participants preferred the paternalistic option, getting a doctor’s subjective recommendation, even if an objectively correct choice did not exist.

The questions probing medical advice were hypothetical, but Kassirer, Levine, and Gaertig find similar patterns in a real-world experiment where subjects could win money, or lose it. They offered participants the chance to take part in one of two raffles, and gave them information about the possible outcomes associated with each raffle to help them decide which best fit their risk appetite. Raffle “experts,” with access to data about past raffles, also weighed in—some with more paternalistic recommendations about what to do, and some who offered further information but no explicit recommendation.

The researchers find that participants most conflicted by the choice responded most positively to experts who provided their personal recommendations. Moreover, if the advice turned out to lead to unhappy results (such as losing money), they didn’t blame the experts. “It seems surprising at first, but less so when you think about the counterfactual,” says Levine. “You might feel angry at your doctor for giving you advice that led to a bad outcome. But you might feel angrier if you felt he sat back and let you make a mistake.”

For Levine, the research resonated with her own experience with a complicated pregnancy, when medical norms made trying to get clear recommendations from a doctor “like pulling teeth.” She sees Americans having some of the same experiences in the back-and-forth over how to loosen lockdowns without endangering public health during the COVID-19 crisis. They know that the experts can’t give definite dates for when certain activities will be safe again—and they might not blame them for that. But they nonetheless suffer if no guidelines are offered at all.

The politicization of expertise

If we cast epidemiologists and clinicians as hedgehogs, who make calls mainly on the basis of their evolving understanding of the coronavirus and COVID-19, who are the foxes that people can turn to for longer-term guidance? Whom can people trust to pursue a reopening of the economy while also keeping in mind virology and biology, people’s social and financial needs, and civil rights? The answer quickly turns political: in the US, some conservatives argue that Democrats have ceded too much power to narrowly focused bureaucrats; many on the left see US president Donald Trump as a fox manqué, pretending to synthesize science he does not understand.

These dueling political perspectives have consequences. A team of researchers—Columbia’s Andrey Simonov, Columbia PhD candidate Szymon Sacher, Chicago Booth’s Jean-Pierre Dubé, and University of Washington’s Shirsho Biswas (a recent graduate of Booth’s PhD Program)—looked at Nielsen television viewing numbers to test for and measure a causal effect of Fox News on compliance with state and local stay-at-home orders between March and July. They measured social distancing using location data from millions of cell phones. To identify the causal effects, they exploited the random assignment of channel positions across US cable markets. Viewers in general are more likely to watch a channel that is high on the dial (such as 1, 2, or 3) rather than lower (101, 102, or 103). Using only the incremental viewership induced by a local channel position in an individual zip code, the researchers find that watching Fox News caused people to social distance less. An additional 30 minutes of Fox News per week caused between 5 and 28 percent of the persuadable audience not to comply with social distancing.

“Fox News has a very large persuasive effect,” says Dubé. What if all the experts got their advice wrong, and social distancing was shown to be useless? Even in that unlikely event, he says, it’s still striking that people would have ignored expert advice coming from the World Health Organization, the CDC, and medical-school faculty in favor of what Fox News anchors were saying.

“I want to know if this is good or bad for society,” he says. “The harm it creates for society is Fox News causes people to distrust experts.” (For more, read “Fox News Causes Viewers to Disregard Social Distancing.”)

Seth Masket, a political scientist at the University of Denver, meanwhile notes that COVID-19 deaths and coronavirus case numbers tend to be less predictive of when a state opens up than are either the governor’s political party or the state voting patterns for the 2016 presidential elections.

Institutions of higher education aren’t immune to party politics. Data collected this spring by the Chronicle of Higher Education reveal that whereas just 45 percent of colleges and universities in states won by Hillary Clinton in 2016 were planning in-person classes for fall 2020, the portion climbed to 80 percent in states won by Trump.

Researchers at Davidson College in North Carolina, led by Christopher Marsicano, demonstrate a similar trend in colleges’ transition to online teaching in March, as the scale of the pandemic was becoming clear. Colleges in states led by Republican governors were slower to stop in-person classes than those in Democrat-led states, independent of campus infrastructure, class size, or ratio of students living on campus. Neither analysis of colleges controlled for coronavirus cases, though the Davidson team points out that decisions to go online usually preceded a confirmed case of COVID-19 at the school. “While the threat of the virus is very real, imminent threats of members of campus communities contracting the virus likely did not end in-person instruction,” they write.

Weber’s work in Europe suggests that leaning right politically has a relationship with skepticism toward experts, albeit a weak one. Other factors such as intellectual interests, risk appetite, and trust in people generally are much stronger predictors.

And while their political views may be responsible for people’s skepticism toward experts, research suggests people are capable of trusting politicians who disregard experts without necessarily disregarding those experts themselves. Booth’s Levine finds in a series of studies on prosocial lying, including with University of Pennsylvania’s Maurice E. Schweitzer, that people can trust someone while at the same time believing that person is a liar. “There are different kinds of trust,” Levine says. “We can trust in someone’s benevolence and goodwill while believing they are lying to us. Many people who support Trump might not believe what he’s saying while still believing he is fighting for them.”

Personal experience, moreover, can move people toward taking expert advice. In a study set in the US, University of Texas’s Olivier Coibion, University of California at Berkeley’s Yuriy Gorodnichenko, and Weber used the staggered introduction of coronavirus lockdowns to examine their impact on consumers’ spending habits, economic expectations, and trust in institutions. They find that, even controlling for how people voted in the 2016 election, being under lockdown increased a person’s esteem for the CDC. “I’d guess people were really scared of the situation, and looking for guidance and leadership,” says Weber. “They found it in the CDC.”

Recognizing knowledge and remaining humble

Science, however, evolves—as does expert opinion, which it informs. The CDC updated its guidance about COVID-19 as health professionals gained more information and evidence. But it takes time for such evidence to accumulate, and then for early research findings to be reviewed, tested, and ultimately accepted as bankable knowledge—or rejected and replaced by other hypotheses. Thus, it’s not necessarily antiscience to be skeptical of new information, argues one economics expert—just a recognition of the scientific process.

Booth’s Kevin M. Murphy says it’s important to distinguish between short-term news and long-term, established knowledge. The former is the “flow” of knowledge, which can be provocative but may soon enough be proven incorrect. This includes new ideas that may never have solid support, even if they come from people positioned as experts. The latter is the “stock” of knowledge that is valuable and has taken years to establish.

“Our best answers are likely to come when we can apply proven ideas and principles to new problems. Things often seem ‘unprecedented’ when we look at them in terms of the details but much less so when you think harder,” he says. “To me, the key in applying economics is to see the commonality with problems you have studied before.” What can we learn from past problems, and existing ideas? What does that say about the current situation, including the limitations of new ideas? Murphy sees new theories as a last resort, and at convocations he has counseled new graduates to think “inside the box”—not just out of it, although that may be fashionable.

Of course, when time is of the essence—such as in the midst of a global pandemic, when scientific understanding changes quickly—there may be value in new data and ideas. But that is when it is particularly important to depend on established knowledge, he maintains. A theory created to address the current challenge may be enticing but also shaky when time is most precious.

“Given all of this, I think it is important to be humble up front,” he says. “We need to be clear that we are still learning about the situation and that our ideas may change. To maintain confidence, we should avoid the most speculative predictions and earn people’s confidence through honesty and making clear the limits of our expertise. We know some things but not everything. We often forget that policy decisions must consider more than our expertise. Most importantly, we should view what we provide as input not as answers. We often say that people who disagree with us ‘are not listening,’ but they may be listening and just disagree.” Experts make mistakes, and to say or imply otherwise undermines their credibility.

University of Chicago’s and Booth’s Lars Peter Hansen, a Nobel laureate, explains in a Chicago Booth Review essay how to make sense of new information as it comes in. Data, he writes, though vital to scientific understanding, are also frequently open to interpretation. “While we want to embrace evidence, the evidence seldom speaks for itself; typically, it requires a modeling or conceptual framework for interpretation. Put another way, economists—and everyone else—need two things to draw a conclusion: data, and some way of making sense of the data.” He and others seek to model how to “incorporate, meaningfully acknowledge, and capture the limits to our understanding,” and to understand the implications of these limits. (For more, read “Purely Evidence-Based Policy Doesn’t Exist.”)

It can be extremely valuable, he writes in a different essay, for decision makers to be confronted with existing scientific evidence from multiple disciplines, at the same time respecting the limits of this knowledge, in order to make informed decisions in the moment. In many settings, there is no single, agreed-upon model but rather a collection of alternative models with differing quantitative predictions. Take epidemiological models enriched to confront both health and economic considerations, for example, especially those used to inform policies such as lockdowns. “I worry when policy makers seemingly embrace one such model without a full appreciation of its underlying limitations. It can be harmful when decision makers embrace a specific model that delivers the findings that they prefer to see,” writes Hansen, expressing hope that we can stop pushing model uncertainties into the background of policy making. Hansen sees potential promise in efforts to integrate economics and epidemiology when building models aimed at guiding efforts to confront health and economic trade-offs in future pandemics. But there are knowledge limits in both fields that should be recognized when using such “integrated assessment models,” he says. (For more, read “How Quantitative Models Can Help Policy Makers Respond to COVID-19.”)

The gap between epidemiologists and economists, as between nonexperts and experts, can be wide—but much is to be gained by bridging it. Experts in any field can make mistakes by failing to recognize the reality in which others live, which affects how their advice will land.

Sapienza and Zingales conclude their paper on the divergent opinions of experts and nonexperts not by wagging fingers at ordinary Americans for not getting on board with expert opinion, but by urging experts to take off their professional blinders. “The context in which these [expert opinion poll] questions are asked induce economists to answer them in a literal sense,” they write. “Hopefully, the same economists, when they do policy advice, would answer the same questions very differently.”

Similarly, policy makers and the media can try hard to ask their questions from new angles, and with context in mind, much like the New York Times did in May in asking epidemiologists not when they think it will be safe to send children to school or step on an airplane—questions that alone might yield lots of “when there is a vaccine” responses. Instead, the newspaper asked epidemiologists when they plan to do these things themselves, forcing them to consider a wider range of factors than those contained in their area of expertise.

Still, as policy makers look not just to virologists, but economists, sociologists, historians, and others for guidance on how to cope with this pandemic and its fallout, responses couched in caution and hedged for public consumption may feel inadequate. Here, Penn’s Tetlock offers some succor. His team’s efforts to identify superforecasters uncovered several factors that boost performance, and one was public scrutiny. Which is to say, today’s skeptical citizens, if truly paying attention, may be improving the very advice they are scrutinizing.