Why Evidence Doesn't Matter
Given that a good reporter is supposed to be both truthful and balanced, how should journalists cover an issue when opposing camps refuse to agree on the very nature of the truth? (In other words, in this current political climate, how should journalists cover literally any issue at all?)
As psychology professor Derek J. Koehler explained last year in a piece for New York Times Sunday Review, “standard journalistic practice” is to represent both sides of an issue, but when the vast majority of qualified experts share one stance on a topic, while only a tiny minority disagree, giving equal weight to each viewpoint can be extremely misleading.
The BBC has adopted “weight of evidence” reporting, which attempts a compromise by giving airtime to experts with both perspectives, while also being careful to inform the public that one school of thought is far more widely accepted. However, Koehler had his suspicions about how effective this really was, so he decided to put it to a test.
Koehler had one group of subjects read a brief numerical overview of a discussion among experts at a University of Chicago panel on economic issues, which broke down how many panelists took each stance on a given subject. Koehler’s other group got the same information, along with a quote from an expert in each camp, backing up their opinion. For some issues, opinions were more or less evenly divided. But a discussion about a carbon tax found 93 economic experts in favor, 5 undecided, and just 2 opposed.
Unfortunately, Koehler found that just reading a quote from one of those two anti-carbon taxers was enough to deeply confuse his experimental group. People who got those additional quotes were more likely to consider the discussion unresolved, and less likely to believe there was enough consensus to guide policy.
Koehler suggests a few possible explanations: seeing quotes from participants on each side allows us to dramatize a debate in our heads—and it’s easier to picture that exchange as a one-on-one showdown, instead of ninety-three-on-two. It could also come down to the human brain’s blind spot when it comes to accurately weighing proportions. Or it could be that our brains are quick to pick up on any perceived conflict, without pausing to run the numbers.
Factor in the conflict-hungry 24-hour news cycle, and maybe it’s no wonder that no amount of experts seem capable of getting us all on the same page. British-born comedian and Last Week Tonight host John Oliver would agree.
On a May 2014 segment, Oliver lamented a study that found roughly a quarter of Americans were skeptical of manmade climate change--in contrary to an overwhelming majority of experts and mountains of data.
“And I think I know why so many people think it’s open to debate,” he added, “because on TV, it is.” Oliver criticized cable news shows for staging countless one-on-one showdowns between a climate change skeptic and a climate change believer (usually Bill Nye). If just reading a quote is enough to get Koehler’s subjects swept up in the heady emotions of a war of the words, you have to wonder about the effect of staring into the angry faces of two yelling men in bowties.
“Weight of evidence” would be a handy quick fix—if it worked. Unfortunately, it looks like there are no quick fixes in sight. So until journalism discovers some remarkable new innovation, perhaps the fairest, most accurate way to discuss climate change is the one John Oliver used to close out that May 2014 segment—accent not required.