Laurie Wang

New research from Yale’s Department of Psychology assesses the impact of third-party fact checking warnings on a user’s ability to differentiate between real and fake news stories on social media websites like Facebook.

Two researchers submitted a paper for publication on Sept. 12 in which they presented participants with real and fake news articles and tested how well those participants could distinguish between them. A control test contained a mix of unmarked real and fake news headlines and sources, while in a second test some stories included warnings that third-party fact checkers had disputed their accuracy. The researchers then compared the results by the age and voting affiliation of participants to see if different demographics performed differently.

“The key take-home from our study is that when some stories are tagged as ‘disputed,’ those studies are believed less — even by Trump supporters,” corresponding author David Rand said. “However, [tagging news as ‘disputed’] also makes many people trust all the other stories which don’t have warnings more.”

Gordon Pennycook, a postdoctoral fellow in psychology and the first author on the study, said this second finding was the more surprising of the two. He said that he and Rand have termed it the “implied truth effect,” since when participants were shown a group of news stories in which only some were disputed, they tended to view untagged news as accurate, even if it was false.

“The bad thing about it is that the effect is stronger, at least for Trump supporters and younger adults,” Pennycook added, citing previous research he did with Rand showing Trump supporters to be more susceptible than Clinton supporters to fake news. He added that younger adults tend to spend more time on sites like Facebook.

The two researchers recruited over 7,000 participants over the course of their experiments — something Pennycook said was not common among psychological studies, since most only recruit around 50 participants. He added that the large sample size helped him and Rand accurately measure the size of the effects they observed and boosted their confidence in their findings.

“We used a much bigger sample than usual because we wanted to be very sure that our results were robust, given the real-world relevance of the findings,” Rand said.

Although misleading stories have long existed in tabloids, there is a difference in scope and context between seeing a tabloid in its section at a grocery store and seeing a fake news story that a friend shares on Facebook, Pennycook said. Additionally, he said, the sheer amount of fake news on the internet makes it difficult to vet every story.

“Given that it’s much easier to produce fake news than to fact check it, it’s likely only a tiny subset of stories will be successfully tagged,” Rand said. Indeed, the paper cites Brandolini’s law in its discussion, which states that the amount of energy needed to refute falsehoods is an order of magnitude bigger than that required to produce it.

While each news story in the study contained a photo, headline, lede and source website to mimic how Facebook displays news stories, Pennycook said the study’s methodology may have differed from most users’ experiences, since the researchers could not use Facebook’s actual platform to conduct the experiments and instead had to rely on an imitation.

While the researchers are waiting for their submitted paper to be peer-reviewed and published, Pennycook said, they are working to develop more effective intervention methods to debunk fake news, as well as working to understand why some people might trust inaccurate news sources.

Pennycook said future research should look at the impact of right-wing, anti-media bias on that population’s higher likelihood of trusting fake news stories.

“To discern real from fake, you need to have some sort of trust in real, actual news sources that have journalistic standards,” he added.

Ironically, the study has been misconstrued by both liberal and conservative media outlets, Pennycook said. A Politico headline stated, “Tagging fake news on Facebook doesn’t work, study says,” when in fact the study did show that including third-party fact checker warnings made it easier for participants to distinguish between real and fake news. Pennycook added that both The Atlantic and Breitbart ran stories that required corrections.

Over two billion people use Facebook every month, according to the company’s website.

Madeline Bender | madeline.bender@yale.edu