Caption: Image of famous “FACTS NOT OPINIONS” motto at the Kirkaldy Testing Works (Wikimeida Commons)
So you took the Pro-Truth Pledge (PTP), or are thinking about taking it, and are wondering what we mean by facts, opinions, and experts? Great questions, and very relevant for two behaviors of the pledge:
- Clarify: distinguish between my opinion and the facts
- Defer: recognize the opinions of experts as more likely to be accurate when the facts are disputed
Just the Facts
The main purpose of the PTP is to fight misinformation, and we take our approach to the facts from that perspective. We aim for a shared definition of “facts” on which all reasonable people can agree, including 1) physical phenomena we can reasonably observe with our senses, and 2) abstract phenomena we can reasonably derive from a few basic principles of logic, math, and other disciplines. Let’s consider the first, using the example of a basketball.
Caption: Image of basketball (Timothy Takemoto/Flickr)
We can all agree that a typical basketball is orange, round, makes a bouncy sound, and smells and tastes rubbery. That uses all five of our senses: sight, touch, hearing, smell, and taste (please don’t taste basketballs). We aren’t interested in the semantics of the deeper meaning of “orange” or “smells of rubber”: all we are concerned is that reasonable people can reasonably agree on these aspects of “basketball.” While we sympathize with people who have color blindness or synesthesia or other conditions that makes it difficult for them to trust their senses, our physical sensory experiences are one of the two best tools we have for a shared understanding of reality, which is why we use them as one definition of “fact.”
The way this definition becomes relevant in public discourse, which is the area covered by the PTP, stems from our ability to observe with our senses claims made by politicians. For instance, we have photographic evidence that Hillary Clinton did not land under sniper fire in Bosnia, unlike she claimed. On the other side of the political aisle, videotaped and photographic evidence shows that Donald Trump’s claims that he had the biggest inauguration size ever are incorrect. In both cases, we relied on our physical senses to evaluate these claims.
The other category we term “facts” refers to abstract phenomena derived from logic, math, and other related disciplines. Thus, reasonable people can agree that 2+2=4. We can agree that if a>b and b>c then a>c. We can agree that 50% of 100 is 50. We can agree that, following the guidelines of logic and probabilistic thinking, the more outlandish the claim, the more evidence it requires, and “extraordinary claims require extraordinary evidence,” a phrase popularized by Carl Sagan. We can also agree that if someone has a clear financial or other motivation that drives their claims, their claims should be considered less weighty. Likewise, sources that are frequently biased in one direction or another should be trusted less than more neutral sources on topics that align with the bias of the frequently-biased source. Moreover, sources that are more fact-based deserve higher trust than those which rely more on opinions.
Caption: Meme saying “Not sure if this is the time to be super-picky or if i should wait until later in this blog” (Made by blog author)
Just don’t go all lawyerly and start being super-picky. We’re talking about general principles on which reasonable people can agree. Likewise, we’re not trying to make an exhaustive list here, just listing a few examples. Any of these can be exhaustively debated in philosophy classes, but that’s not the point of the PTP.
Now going on to public policy relevance, we can see that if someone makes a claim that millions of non-citizens voted illegally in the 2016 election and does not show evidence despite repeated requests for evidence, while plenty of transparent studies show clear evidence of negligible voter fraud in US elections, the claim of millions of illegal voters can be safely judged as false. That was a claim made by a Trump supporter, and endorsed by Trump himself. Clinton, in turn, made a claim that she lost in Wisconsin because “200,000 people in Wisconsin were either denied or chilled in their efforts to vote,” an extraordinary claim for which she did not provide appropriate evidence.
We acknowledge that the two categories above are a rather narrow understanding of “facts.” In other contexts, facts might include one’s internal mental experiences, such as feelings of pleasure or pain, or one’s thoughts about a topic, such as whether a politician changed her mind based on actual evidence or just to be elected. However, those internal mental states are inaccessible to external observers, and we tend to overestimate our ability to know other people’s subjective experiences. Since the pledge is meant to prevent misinformation, we want to have a narrow definition of facts. Our aim is ensuring that when someone who signed the pledge is accused of making a false statement, we can either reasonably observe the false statement with our senses, or reasonably derive it from principles of logic, math, and related disciplines.
Caption: Meme saying “So you have an opinion… tell me more” (Created by blog author)
One of the behaviors of the PTP involves differentiating your opinion from the facts. By opinion we refer to anything that is an evaluation of a situation, whether a personal opinion or an expert analysis. Let’s start with something easy: “I think you stink” is an opinion. By contrast, “I smelled your body odor from about 30 feet away” is an observation of fact. We’d want to get other noses in there to verify the fact, but you get the gist. Another fact: “it’s 70 degrees outside, according to the reading on the thermometer.” An opinion would be “yeesh, that’s hot” or “brrr, that’s cold.” The US federally-mandated minimum wage is $7.25: that’s a fact, as of 11/8/2017. An opinion might be that it’s too low, too high, or just right. Holding everything else steady and cutting the income tax would result in you paying less taxes: whether doing so would create more jobs is an opinion.
To continue our use of Clinton and Trump, many people who dislike Clinton claim that it’s a fact that she’s a criminal and needs to be in jail. In turn, many people who dislike Trump claim that it’s a fact that he has colluded with Russia to undermine the US election. In both cases, what these people hold are opinions, rather than facts. Despite a long investigation, Clinton has not been convicted of a crime or even charged with one, while the investigation into Trump’s potential collusion with Russia is ongoing at the time of this blog post’s writing. To continue treading these somewhat dangerous waters, some people claim that it’s a fact that Bernie Sanders is not a “real Democrat.” Of course, their claim depends on their own opinion of what makes someone a “real Democrat.” Someone else might claim that Trump is not a “true conservative,” which again depends on one’s opinion of what makes someone a “true conservative.”
As we can see from these examples, opinions are often subjects of disagreement among different people. However, even when judgments about reality are subjects of common agreement, they still fall into the realm of opinion. For example, you’ll find few people in Columbus, OH, where I live, who would not describe a temperature above 100 degrees Fahrenheit as “hot” or who think that we should not have any taxes and just fend for ourselves. Still, despite our common agreement about temperature or the benefit of government functions, these areas of common agreement are still opinions. Remember, your opinions are opinions even when they are widely shared: as a side note, be suspicious when you perceive your opinion to be widely shared, as we tend to overestimate the extent to which other people share our opinions.
So make sure to enact the pledge by recognizing the difference between the facts and your opinions. If it might be unclear to others whether your statement is a fact or opinion, clarify this matter. Minimize editorializing, meaning mixing in your opinion with facts, such as in “our currently too-low minimum wage is $7.25” or “our government’s incompetent UN Ambassador is heading to a meeting in Brussels.” Both of these statements combine opinion with facts, and make it unclear which is which: it’s best to avoid such statements. We have a relatively high standard of what constitutes a violation of the pledge, and only the more extreme forms of such editorializing that convey information in an obviously deceptive manner would qualify as violations. Still, I would encourage all pledge-takers to orient toward fully enacting all the behaviors of the pledge, as opposed to doing the bare minimum to not violate it.
Caption: Meme saying “one does not simply disregard the opinions of experts” (Made by blog author)
Any one of us can have an opinion on any topic. We can have the belief that chocolate ice creams is better than vanilla, or that baseball is boring and basketball is not, or that cats are better than dogs, or any other opinion. In the realm of public discourse, we can hold the opinion that our taxes are too high or too low, or that one politician is better than another, or that the government should do more or less to cover people’s medical bills, or that our gun policy is too strict or too loose. Anyone’s opinion, everything else being equal, has just as much validity as all other people’s opinions from the perspective of truth.
However, everything else is often not equal, since some people have more expertise than others. Thus, the Pro-Truth Pledge asks signers, as one of the behaviors, to “recognize the opinions of experts as more likely to be accurate when the facts are disputed.” Let’s unpack this behavior.
First, the term “expert:” what does that even mean? Well, “expertise” can refer to many different things. For example, say I spend 30 minutes researching the best Android phones of 2017, which I did yesterday after my phone started acting up and I decided to get a new one. That research gained me some expertise. Thus, my opinion about what are the best Android phones of 2017 will likely be closer to reality than another person who did not research this matter and whose activities in general are not related to smartphones.
However, that’s not what we mean by “expert.” We mean someone who has a quite significant familiarity with a specific topic area, as shown by commonly-recognized credentials such as a certification, an academic degree, publication of a book, years of experience in a field, or other way that a reasonable person may recognize an “expert.” Thus, an expert in Android phones might be someone who has worked for several years in making these phones, or an experienced salesperson, or a technology writer. They would be able to hold a lengthy conversation on the specs of smartphones, describe why certain people might prefer one or the other, and how to determine which is right for you.
Experts like that are able to draw on their substantial body of knowledge and experience to provide an opinion – often phrased as “expert analysis” – that is, as the pledge states, “more likely to be accurate when the facts are disputed.” That doesn’t mean an expert will always be right, simple more likely to be right when the facts are disputed, following probabilistic thinking. So for policy expertise, a geologist studying well water issues is more likely to be closer to reality in evaluating hydrofracking than someone who has not studies well water for years; an economist specializing in taxes will be more likely to be correct about the outcomes of proposed changes in taxation than someone who is not; a foreign policy expert in North Korea will be more likely to be right about the reactions of the North Korean leadership to any given external event than someone who is not.
This greater likelihood of experts being closer to the truth when the facts are disputed explains why the pledge encourages pledge-takers to defer to experts. However, this is not an absolute principle by any means. First, research shows that experts do best in evaluating reality in environments that are relatively stable over time and thus predictable, and also when the experts have a chance to learn about the predictable aspects of this environment. Thus, an expert on North Korea may be less able to make an accurate evaluation in the context of a regime change, since her previous good read on the leadership is now less salient with a new leadership. Likewise, an expert on taxation might have less capacity to predict the impact of taxation on newly-emerging fields like cryptocurrency. Second, other research shows that ideological biases can have a strongly negative impact on the ability of experts to make accurate evaluations. Third, financial and other material motivations can sway experts to conduct an analysis favorable to their financial sponsor. Other factors may also cloud expert judgment.
Thus, while we recommend that pledge-takers by default defer to expert judgment as more likely to be accurate, we do not demand it, except in two limited cases. One is in expert evaluations by credible fact-checking organizations, as described in this post, and another is in the scientific consensus. Credible fact-checking organizations hire experts whose expertise is in unearthing the facts and comparing the facts to statements made by public figures, to evaluate the accuracy of the statement. Their financial motivations and field of expertise align well to ensure they focus on figuring out the truth of reality, and their reputation is maintained if they do a good job. Since we consider “credible” only fact-checking organizations that have been vetted by the Poynter Institute International Fact-Checking Network, we are comfortable with asking pledge-takers to abide by the evaluations of these fact-checkers.
Similarly, while individual scientists may make mistakes, it is incredibly rare for the scientific consensus as a whole to be wrong. Scientists get rewarded in money and reputation for finding fault with statements about reality made by other scientists. Thus, for the large majority of them to agree on something – for there to be a scientific consensus – is a pretty clear indicator from a probabilistic perspective that whatever they agree on reflects reality accurately.
That doesn’t mean that credible fact-checkers can’t be wrong, and neither does it mean that the scientific consensus can’t be wrong. What it does mean is that going against credible fact-checking organizations and the scientific consensus will be very, very likely to be wrong. Let me phrase it this way: I’d be happy to stake a $100 to $1 bet on the accuracy of the scientific consensus or a credible fact-checking organization. Because of this high likelihood of losing if you bet against the scientific consensus or a credible fact-checking organization, we perceive such bets to be motivated not by a search for truth but biased motivations. Therefore, we ask that pledge-takers do not make such bets and abide by the scientific consensus and credible fact-checking organizations as part of taking the pledge.
The only exception is a scientist in a field relevant to scientific consensus, who we welcome questioning the consensus, as that’s the way progress is made in science: this scientist is much less likely to be questioning the consensus from biases motivations, rather than the pure drive for advancing scientific knowledge based on new information and insights.
I hope these guidelines help you see where we’re coming from when we talk about facts, opinions, and experts from the perspective of the pledge. Let us know what questions you have!
6 thoughts on “Facts, Opinions, and Experts”
I consider the AAAS to be a credible fact-checking organization, and rarely disagree with their positions.
However, their stated consensus on GMO health safety presents an incautious level of certainty that appears unusual, perhaps even unique in their history. This level of certainty appears to have been bolstered by influences outside legitimate scientific investigation. Given the number, type, and significance of unexplained risk indicators, the AAAS’ position statements must be judged insufficiently supported.
Neither reversing the burden of proof, nor pointing to irrational opposition to revolutionary GE technology provides support.
Several factors call for skepticism: the lack of independent data, absence of evaluation criteria developed by safety standards experts, research restrictions beholden to economic needs, the amount of investment by industry in preventing open research and access to source data, and the implausibility of any perfectly risk-free technology.
For these reasons, I believe a well-informed search for reliable facts and information obligates us toward skepticism on their consensus position until these deficits with that position’s justifications are properly addressed.
While I am not familiar with AAAS or with their stance on GMO health safety, it is the policy of the Pro-Truth Pledge project to use scientific consensus as a guide in evaluating whether any particular public statement is aligned with the pledge. The PTP leadership team does not have expertise in any particular scientific discipline (except for behavioral science, which is at the core of our work). We don’t want to second-guess science experts or presume that we can do better than them in trying to determine whether a controversial statement about a scientific discipline is true. When we asses statements made by public figures we want to remain humble as we try to determine what is true, and we have a strong probabilistic estimate that scientific consensus on any particular issue is most likely to reflect the truth. While scientific research may be politicized or otherwise affected by non-scientific factors, it remains our best way to determine what is true. Scientific consensus is most likely to be true and least likely to be wrong as compared to other methods for objectively assessing reality.
controversial statement about a scientific discipline…Hmmm
If you don’t understand the basic Science and Math, stay out of the issue. Sorry, Al Gore has a US Government degree, he is not someone I listen to about Science.
Maybe Political Science, but that is obscuring things. But one recent petition signed by thousands of “Scientist’s”, was signed by mostly Political and Social Scientists.
That might hold sway with HS kids, but not those of us who know their way around a Lab.
THIS is a good reference to use when reviewing Climate Science Graphs
5 Ways Writers Use Misleading Graphs To Manipulate You [INFOGRAPHIC]
“In this post-truth era, graphs are being used to skew data and spin narrative like never before. Especially with the velocity at which some of these topics spread across social media.
All it takes is a single graph from a less-than-reputable source, blasted out to a list of followers, to spread a false narrative around the world.”
“The data doesn’t even have to be bad–it could just be presented in a misleading way. ”
I mean, there is a whole Wikipedia page, Reddit community, and hundreds of articles about how graphs can be used to misinform readers.
In fact the Wikipedia PAGE on misleading graphs, is a copy of what NASA does with the CO2 Graph!!
A truncated graph (also known as a torn graph) has a y axis that does not start at 0. These graphs can create the impression of important change where there is relatively little change.
create the impression of important change where there is relatively little change.
CREATE THE IMPRESSION…..SELL IT, in other words!!
of important change where there is relatively little change.
Give you the Impression of IMPORTANT change, when there IS RELATIVELY LITTLE CHANGE!!
Look at the difference in the graphs on Wikipedia, THAT is the same level of truncation that the CO2 Graph has!!
SEE how much different THEY LOOK??
MATH is the language of Science, Scientist’s need to use it HONESTLY
Baseball is boring and basketball is not?? What!! Oh my, Gleb! Baseball is so much better. Oh and by the way, cats ARE better than dog because dogs can’t run an MRI machine, but cats can! 🙂
I can neither confirm nor deny my personal opinion that cats are better than dogs 🙂