Fact vs Opinion

During a recent long drive, my husband and I listened to some podcasts. One of them was an interview with a researcher from the Pew Research Center, talking about a recent survey Pew has conducted to determine Americans' ability to distinguish factual statements from opinion statements.

The differentiation between factual and opinion statements used in this study – the capacity to be proved or disproved by objective evidence – is commonly used by others as well, but may vary somewhat from how “facts” are sometimes discussed in debates – as statements that are true.¹ While Americans’ sense of what is true and false is important, this study was not intended as a knowledge quiz of news content. Instead, this study was intended to explore whether the public sees distinctions between news that is based upon objective evidence and news that is not.

The statements presented were political; five were written to appeal more to Democrats and five appealed more to Republicans. A lot of testing was done beforehand to make sure that respondents understood exactly what was being asked of them:

In the survey, respondents read a series of news statements and were asked to put each statement in one of two categories:

1. A factual statement, regardless of whether it was accurate or inaccurate. In other words, they were to choose this classification if they thought that the statement could be proved or disproved based on objective evidence.

2. An opinion statement, regardless of whether they agreed with the statement or not. In other words, they were to choose this classification if they thought that it was based on the values and beliefs of the journalist or the source making the statement, and could not definitively be proved or disproved based on objective evidence.

So what were the results?

...a majority of Americans correctly identified at least three of the five statements in each set. But this result is only a little better than random guesses. Far fewer Americans got all five correct, and roughly a quarter got most or all wrong.

Do you find that disturbing? I do. But I'm not surprised, either.

The people who did the best on the test were those with a high level of political awareness (36% identified all five factual news statements) and those who were "digitally savvy" (44% identified all five opinion statements correctly.)

 And though political awareness and digital savviness are related to education in predictable ways, these relationships persist even when accounting for an individual’s education level.

And as you can imagine, political bias also played a part in skewing the responses. This was shown when respondents were matched with their party affiliations, and made clearer when they were asked if they agreed or not with the statements they called "opinion." But that really wasn't the entire point of the survey -- it was to determine if the respondents could distinguish between a factual statement and an opinion at all. That's a critical thinking skill. For instance:

Overall, Republicans and Democrats were more likely to classify both factual and opinion statements as factual when they appealed most to their side. Consider, for example, the factual statement “President Barack Obama was born in the United States” – one that may be perceived as more congenial to the political left and less so to the political right. Nearly nine-in-ten Democrats (89%) correctly identified it as a factual statement, compared with 63% of Republicans. On the other hand, almost four-in-ten Democrats (37%) incorrectly classified the left-appealing opinion statement “Increasing the federal minimum wage to $15 an hour is essential for the health of the U.S. economy” as factual, compared with about half as many Republicans (17%).

and further:

When Americans see a news statement as factual, they overwhelmingly also believe it to be accurate. This is true for both statements they correctly and incorrectly identified as factual, though small portions of the public did call statements both factual and inaccurate.

 

This is a failure of the educational system, and it certainly has a profound effect on electoral politics, as well as explaining some of the reasons why we're in this current predicament. People without critical thinking skills can be easily and successfully manipulated, as they are now being, by politicians, foreign governments, and biased media. The truly frightening part is how long it would take to correct such a failure - and how little political will there is, and has been, for an educated, critical, and independent electorate.

If you want to try the test yourself, it's online here.

 

Comments

[Kathy Hughes]

So interesting Beth. Glad to say I got them all correct but pretty frightening to see the overall statistics. Such a sad state our country is in - but worse is looking into the future.
Would be very easy to disconnect for our own mental health. Thanks as always for your clarity and is nice to read your unbiased opinions. Hope you and J. are well!
xxoo

[Kurt]

Fun exercise. I sailed through but did have to pause on one to think it over. While it's right to point to our educational system as one of the culprits here, I'm more concerned about those who aren't MERELY ignorant but are WILLFULLY ignorant. Many who fall into the latter category are close to power or even wield it.

[am]

Thank you, Beth.

[Duchesse]

I missed one, but I protest. (Kathy Hughes, is it the same one?) #10: ISIS lost a significant portion of its territory in Iraq and Syria in 2017. "Significant" in this sentence does not refer to statistical significance and so I called it as opinion. It's a classic political weasel word. Try the word in a non-political statement like, "My wife lost a significant amount at poker last night" and you will see why I protest my score :).

I agree that critical thinking skills have eroded. There's also the other side of the coin, "facts" that are lies presented as facts. The USG's lies about the Vietnam body counts brought me to Canada in 1970.

[Natalie]

If a similar test was given to random members of the public almost anywhere in the world I wonder if results would differ a great deal? Ignorance, indifference, lack of attention, distraction, media overload, susceptibility to brainwashing by propaganda and fake news - all these are pretty much the universal condition at present, even if a certain proportion of Americans seem to typify it.

[Roderick Robinson]

You're right not to be surprised. Overhear a discussion about a fictional TV series and, as often as not, it will rapidly become apparent that those speaking regard the "characters" as living people. Their motives will be argued, their past existence analysed, their personalities adumbrated. This is not to say that academics may not use such methods but the approach in those instances is quite different - detached and impartial. There's a strange naivete about the former group, as if their familiarity with the stories had turned them into participants in the drama. Among other things it leads to suggestions that actors such as Bill Cosby (pre la chute, of course) should be elected president.