Author Information: Benjamin T. Rancourt, University of Massachusetts, brancour@philos.umass.edu
Rancourt, Benjamin T. “The More Evidence Heuristic.” Social Epistemology Review and Reply Collective 5, no. 6 (2016): 27-41.
The PDF of the article gives specific page numbers. Shortlink: http://wp.me/p1Bfg0-32b
Editor’s Note:
- The SERRC thanks the contributors and participants—especially William Tuckwell—at the Tartu Graduate Conference in Social Epistemology, at the University of Tartu on 26-27 March 2016, for allowing us to publish selected papers. We will bring these papers, and subsequent replies, together in a special issue of the SERRC.
Image credit: Stanley Sagov, via flickr
Abstract
If A confirms H and B confirms H, it seems reasonable to infer that A&B confirms H. However, this inference is not valid; it is only a heuristic. I show that the level of confirmation A and B each give to H by itself implies nothing about the level of confirmation that A&B gives to H. Any combination of values is possible for P(H), P(H|A), P(H|B) and P(H|AB) is possible. Still, I show the heuristic leads from true premises to true conclusions whenever A and B are statistically independent or correlated, which includes most situations where we evaluate evidence. There is risk, though, when one does not consider how A and B interact; one can miss anti-correlations that lead the heuristic to fail. The problem is exacerbated when groups of people are motivated to support a favored hypothesis. Each person finds evidence that confirms the hypothesis and contributes that evidence to the group’s discourse (e.g. through a climate change denial web site). The temptation to focus solely on the number of individual pieces of evidence can lead people to ignore whether the evidence fits together. In combination with cognitive biases, this can lead to serious errors.[1]
Mary is wondering whether John is a member of the Capricorn Club. Before investigating John, she learns that shopping at Money Stompers is good evidence that someone is a Capricorn Club member. She also learns that drinking at Blotto Bar is good evidence that someone is a Capricorn Club member. After investigating John, Mary finds that he shops at Money Stompers and drinks at Blotto Bar. She concludes that John is probably a Capricorn Club member. Mary makes what seems to be a reasonable inference: from the fact that two propositions each by themselves confirm a hypothesis, infer that their conjunction supports the hypothesis.
However, this inference is not valid. To illustrate this, let me tell you more about Mary’s background knowledge: she knows that a member of the Capricorn Club can never cross the Tropic of Capricorn. She also knows Money Stompers and the Blotto Bar are on opposite sides of the Tropic of Capricorn. Someone who goes to both places cannot be a Capricorn Club member. Given Mary’s background knowledge, two pieces of evidence that each individually confirm the conclusion of Capricorn Club membership together confirm the opposite. Mary’s failure to consider how her evidence fits together as a whole leads to an incorrect conclusion. The inference from “more individually confirmatory evidence” to “more confirmation” is not valid.
Despite this, it would be a mistake to reject all use of the above inference. In this paper, I argue that we should consider it a heuristic. I call it the The More Evidence Heuristic. Like all heuristics, even if it is not perfect, it is usually effective, leading efficiently from true premises to true conclusions. For example, if during a murder investigation, the police find Chris’s fingerprints on the murder weapon, Chris’s shoe print in blood near the body, and Chris’s face on the surveillance footage, this is good evidence that Chris is guilty. Each piece of evidence by itself provides support, and together, they provide stronger support. That conclusion is reasonable. I argue that this is in fact true in most situations where we consider a variety of different pieces of evidence. The More Evidence Heuristic works especially well when there is a community of researchers working independently without a shared preconceived conclusion.
The More Evidence Heuristic is not effective in all situations, as the Capricorn Club case reveals. I argue that there are identifiable risk factors that increase the chance of the More Evidence Heuristic leading to a false conclusions. There is risk when one reaches a conclusion based on pieces of evidence that each individually support it, but one does not consider how each piece of evidence supports the conclusion. One does not consider how the pieces of evidence fit together. The risk is increased when a group of people engages in searching for scraps of evidence to support a given hypothesis that they all want to be true. The group’s tendency will be to focus on the amount of evidence gathered, rather than how the evidence fits together, which can lead to problems (e.g. climate change denial). The risk also increases in conjunction with errors such as cherry picking and confirmation bias. In each case, the risk is that one is more likely to come across more pieces of evidence that do not cohere, and one is also less likely to notice the incoherence.
In Section 1, I explain why the More Evidence Heuristic usually works in real world situations. In Section 2, I show in more detail how the inference is invalid, and I describe common patterns of reasoning that lead to misuse of the heuristic. In Section 3, I present some methods of avoiding the pitfalls of the More Evidence Heuristic.
(1) The Good Side of the Heuristic
The More Evidence Heuristic is a compelling inference that is difficult to resist. How can this be, if it is not valid? The answer is that, despite its formal invalidity, in most real world situations, the More Evidence Heuristic leads from true premises to true conclusions.
The explanation for this usefulness can be found in the following theorem:
Correlated Confirmation Theorem: If A and B are conditionally independent or correlated given H, and independent or anti-correlated given ¬H, then the conjunction AB confirms H if A confirms H and B confirms H.
This theorem implies that when two pieces of evidence for H are more likely to be true together given the assumption that H is true than under the assumption that ¬H is true, the More Evidence Heuristic works. As it turns out, this condition holds in most real world situations we encounter, given the kind of evidence we gather. The Correlated Confirmation Theorem shows why the More Evidence Heuristic usually works for us.
To prove the Correlated Confirmation Theorem with Bayesian confirmation, we start with the fact that evidence E confirms hypothesis H just in case P(E|H)>P(E|¬H). Assume that A and B each individually confirm H. This implies that:
P(A|H) > P(A|¬H), (1.1)
P(B|H) > P(B|¬H). (1.2)
Next, assume that A and B are independent or correlated given H, and assume they are independent or anti-correlated given ¬H. This gives us:
P(AB|H) ≥ P(A|H)P(B|H), (1.3)
P(AB|¬H) ≤ P(A|¬H)P(B|¬H). (1.4)
Given the assumptions, we can show that AB confirms H. First, since probabilities are non-negative, (1.1) and (1.2) together imply:
P(A|H)P(B|H) > P(A|¬H)P(B|¬H). (1.5)
Now, (1.3), (1.4), and (1.5) together imply:
P(AB|H) > P(AB|¬H). (1.6)
Inequality (1.6) implies that AB confirms H. The Correlated Confirmation Theorem is proven. Whenever propositions that confirm H are independent or correlated under the assumption that H is true, then the More Evidence Heuristic works. (There are some cases where A and B are anti-correlated given H, but AB still confirms H.)
It is common that the evidence we acquire relating to a hypothesis can be broken down into propositions that are statistically independent or correlated, assuming the hypothesis. For example, consider the hypothesis that the results of three fair coin tosses are all heads. The results of fair coin tosses are conditionally independent, and assuming the triple heads hypothesis is true, the results the first and second coin tosses are statistically independent.[2] Learning only that the first of two independent coin tosses is heads confirms the hypothesis that the result is two heads (raises the probability from 0.125 to 0.25). Learning only that the second is heads is evidence for two heads (raises the probability from 0.125 to 0.25). Learning that both the first landed heads and that the second landed heads is better evidence for three heads (raises the probability from 0.125 to 0.5).
Standard Venn Diagrams also provide an illustration of the More Evidence Heuristic at work. Consider throwing a dart at the rectangle in Figure 1 such that there is a uniform probability distribution for where the dart lands. In this case, the probability that the dart lands in any given region of the rectangle is directly proportional to that region’s area. Then, learning only that the dart landed in A is evidence that it landed in H, knowing just that it landed in B is evidence that it landed in H, and knowing just that it landed in both A and B is evidence that it landed in H. The More Evidence Heuristic works. Furthermore, it will always work as long as the regions are ellipses. The areas of overlap can never falsify the More Evidence Heuristic without having a more complex shape (such as the shapes in Figure 2 below).
Figure 1: With a uniform probability distribution over the rectangle, this is a case in which the heuristic yields a true conclusion.
Evidence for crimes works like this, too. A suspect’s fingerprint at the scene of a robbery is evidence he committed the robbery. The suspect being witnessed in possession of the stolen goods is also evidence he committed the robbery. The conjunction of both is evidence that he committed the robbery. This is because assuming the suspect committed the crime, leaving a fingerprint is correlated with possessing the stolen goods, since both are characteristic of sloppy robbery: finding his fingerprint suggests he is careless, which suggests he is more likely the kind of burglar who would be seen with stole goods in public. Also, assuming the suspect committed the robbery, being seen in public with the stolen goods is correlated with leaving fingerprints, for the same reason. Assuming the hypothesis is true, the pieces of evidence are correlated, which is why both together are better evidence than each alone. Many of the pieces of evidence we actually deal with are independent or correlated in this way. We do not often find evidence that supports incompatible ways a hypothesis might be true. Thus, the More Evidence Heuristic will work in many cases.
The fact that the More Evidence Heuristic works when dealing with independent lines of evidence is also important to the nature of science as a distributed activity with different lines of inquiry whose results are independent or correlated. The consilience or convergence of different lines of evidence is often considered crucial in science (consilience was proposed by Whewell, and is defended as a criterion of theory choice in (Thagard 1978), (Snyder 2005) and elsewhere). When Darwin presented his theory of evolution in (Darwin 1975), for example, he presented evidence from multiple sources, including geology, comparative anatomy, biogeography and other fields. The reason these make a strong case together is that each is independent or somewhat correlated with the others. Assuming evolution occurred, the success or failure of any one test is independent from or correlated with the others. When they all turned out to support evolution, this made for stronger evidence.[3]
The fact that the More Evidence Heuristic works so often makes it useful in cases where one only needs to know whether there is any degree of confirmation at all. For example, suppose you are trying to decide whether to leave the apartment, and that it is a good idea to leave if your roommate is not home. Just given the time of day, you are already just confident enough that your roommate is gone, but you have access to cost-free evidence due to being in the apartment. If your roommate’s coat is not hanging by the door, this confirms that the roommate is out. If there is no noise coming from the roommate’s room, this confirms that the roommate is out. Acquiring both of these pieces of evidence just by being in the apartment, one concludes that the hypothesis that the roommate is out is confirmed, which implies that one it is a better idea to go out. Using the More Evidence Heuristic is useful here.
Consider why heuristics are useful: they are more efficient than more exact forms of reasoning. This is the case with the More Evidence Heuristic. There are some propositions that are psychologically easy for us to grasp as single pieces of evidence (e.g. the proposition that there is a fingerprint on a weapon, that the coat is not by the door). We can often determine how one of these evidence propositions bears on a hypothesis (e.g. that a person committed a murder, that a roommate is out). However, determining how the conjunction of several evidence propositions bears on a hypothesis requires working out the potentially complicated dependencies among the individual pieces of evidence. As the number of propositions increases, the ways in which unforeseen dependencies can arise increases. Compared to working out the exact dependencies and probabilities, the More Evidence Heuristic is a simple, fast method to judge whether a given body of evidence confirms a hypothesis or not. It can often lead to true conclusions. The More Evidence Heuristic can be a valuable part of one’s cognitive repertoire. It works because the evidence we tend to come across is often independent or correlated. It is a form of ecological rationality: though not formally valid, it is effective in the circumstances in which people are likely to apply it, in keeping with the view of heuristics defended, for example, in (Gigeerenzer and Murray 1987) and (Gigerenzer 2008).
(2) The Dark Side of the Heuristic
Despite the usefulness of the More Evidence Heuristic, the Capricorn Club example shows that it is not formally valid. (As shown above in the proof of 1.6, it can only lead to false conclusions if A and B are anti-correlated given H, though it does not lead to false conclusion in all such situations). To clarify the extent to which the heuristic fails as a general inference pattern, consider Figure 2. Suppose a dart is thrown in such a way that it has equal chance of landing anywhere within the rectangle. The probability that the dart lands in a given region is directly proportional to the area of the region.[4] Learning just that the dart landed within area A is good evidence that it landed in area H. Learning just that the dart landed in area B is good evidence that it landed in area H. But learning that it landed in area A and landed in area B is good evidence that it did not land in area H.
Figure 2: With a uniform probability distribution over the rectangle, this is a counterexample to the More Evidence Heuristic.
As the following theorem implies, in general, we cannot conclude anything about P(H|AB) given just the values P(H), P(H|A), and P(H|B) (proof in the Appendix).
Any Combined Confirmation Theorem For all consistent sets of propositions {H, A, B} and for all x ∈ (0,1)×(0,1)×(0,1)×(0,1), there exists an infinite set R of probability functions such that for each P(⋅) ∈ R, x =〈P(H), P(H|A), P(H|B), P(H|AB)〉. [5]
Starting with any value for P(H), there can be any combination of the degree of Bayesian confirmation that A, B, and AB give to H (aside from some involving probabilities of 1 and 0).[6] The Any Combined Confirmation Theorem 1 implies that the degrees to which A and B individually confirm H do not put any constraints on the degree to which their conjunction confirms H. To get an intuitive sense for how this is true, imagine varying the sizes of the various parts of Figure 1. The intersecting parts can be changed in size and overlap to make any combination of probabilities and conditional probabilities.
Given this result, let us consider ways in which human reasoning can misuse the More Evidence Heuristic. Some patterns of reasoning are prone to leading into those circumstances in which the heuristic produces false conclusions. One all-too-common error mode is the relentless pursuit of evidence in favor of a beloved hypothesis, without attention to how the evidence supports the hypothesis. This can lead one to focus on propositions that support the hypothesis in incompatible ways, and that overall do not support the hypothesis. We can see how failure to consider how evidence supports a hypothesis led Mary astray in the opening example to this paper. The propositions that John shops at Money Stompers and that John drinks at the Blotto Bar are each evidence that John is in the Capricorn Club. Yet they are evidence in favor of that hypothesis by supporting incompatible ways of being in the club (always being on the North side, versus always being on the South side). By failing to consider what kind of life John has to lead in order to go to both locations, reliance on the More Evidence Heuristic led Mary astray. When the evidence for a hypothesis points in different directions, it becomes less likely that as a whole it supports the hypothesis. The lack of concern for the details is a problem.
For every contingent proposition consistent with a body of background knowledge, there are other true propositions that are evidence in favor of the proposition. Thus, it is possible to pick a false hypothesis and find evidence for it, if one looks hard enough. Assuming that one’s favored hypothesis is false, however, the evidence collected will tend to point in all different directions. Some of the evidence in favor of it cancels out other evidence in favor of it, until the evidence as a whole counts against the hypothesis—though each piece of evidence may individually supports the hypothesis. If one is not paying attention, this is a possible outcome.
For example, there are climate change deniers who say both that there has been a pause in global warming, and that the sun is releasing more energy. These are evidence against anthropogenic climate change in incompatible ways; the first by arguing against the existence of climate change, the second by admitting climate change but finding a different cause. When your only concern is to deny climate change and not to think about how your evidence fits together, this can result. A similar phenomenon is found among JFK conspiracy theorists.
It is common for conspiracy proponents to cite as evidence of a conspiracy both the fact that Oswald had a security clearance when in the marines and the fact that he did not have gunshot residue on his face when arrested. The first is evidence of a conspiracy because it suggests Oswald committed the murder for the some intelligence agency, while the second is evidence of a conspiracy because it suggests he did not commit the murder at all. They support the conspiracy conclusion in incompatible ways. Those who only want to prove a conspiracy may not notice this.
The risk of these errors is even greater when a group of people becomes united by common devotion to a favored hypothesis, whether that is climate change denial, a conspiracy theory, or what have you. The members of the group spend a great deal of time looking for anything that is evidence in favor of their favored hypothesis. Each member finds some evidence that confirms the hypothesis, and they contribute that evidence to the group’s discourse (e.g. at a climate change denial web site, or JFK conspiracy theory conference). There is social pressure to present only evidence confirming the favored hypothesis. The problem is that a distributed network of people who each look at different parts of the evidence to support a pet hypothesis will tend to produce a repository of tidbits from so many vastly different fields of research that it can become too difficult for most people even to grasp how they fit together. It becomes nearly impossible to recognize whether they support the hypothesis in conflicting ways.
Since many of these groups view themselves as truthseekers fighting against all odds to expose a powerful enemy, they feel less pressure to formulate a coherent hypothesis, but are content to gather evidence for a vague conclusion like “conspiracy.” Someone confronting the group’s discourse will often encounter facts that may individually support the conclusion, but that together do not.
A particular version of this problem occurs when contrarians focus entirely on tearing down the mainstream view on some topic. To do this, they gather a collection of anomalies that are hard for the accepted view to explain. For example, in (Keeley 1999), when describing one of the most influential Oklahoma City bombing conspiracy books, Keeley points out that the author, “does not endorse any particular alternative account, but instead raises many issues and questions for the official account.” (p. 114) (See also Oliver Stone’s JFK, the AE9/11 Truth website, or ubiquitous lists of things “the experts” or “the official story” can’t explain about a topic.) Sometimes, they put forward no alternative hypothesis, while other times, they suggest that tearing down the mainstream view establishes the truth of their own hypothesis, without explaining how.[7]
The problem here is that there are infinitely many mutually exclusive alternative hypotheses to the “official story.” If one does not propose an alternative but just gathers anomalies that the official story can’t explain, one will likely find support in contradictory ways. Thus, one is more likely to misuse the More Evidence Heuristic. This type of reasoning is not restricted to the fringes, either. Most of us at one time or another will desperately seek out evidence against a hypothesis. The need to deny something is fertile soil for producing improper uses of the More Evidence Heuristic.
Part of what makes the practice of desperately trying to support a pet hypothesis problematic is that it is motivated reasoning, which allows it to combine with other errors to compound the problem. Two such errors concern us here: cherry-picking and confirmation bias.
First, cherry-picking, also known as the incomplete evidence fallacy, is the fallacy of basing one’s conclusions on a subset of one’s total evidence. In Bayesian terms, cherry-picking occurs when CrAB(H) = Cr(H|A) ≠ Cr(H|AB). One’s total evidence is actually AB, but one’s credence is one’s prior credence conditionalized on evidence A, in violation of the total evidence rule. One is not using all evidence. Combined with the More Evidence Heuristic, one might end up focusing only on a subset of the evidence that one has identified, in particular a conjunction of propositions that each individually support one’s conclusion, ignoring the rest of the evidence. Then, one misuses the More Evidence Heuristic. The results are doubly bad.
Next, confirmation bias covers several tendencies: the tendency to notice evidence confirming one’s preconceptions more easily than disconfirming evidence, the tendency to more easily recall confirming evidence, and the tendency to devote more attention to confirming evidence (this is outlined in detail in (Nickerson 1998)). Two manifestations of confirmation bias are particularly salient here.
First, related to cherry-picking, one manifestation of confirmation bias occurs when one ignores negative evidence (such as studies showing a drug has no effect) and favors confirming evidence (studies that show the drug to be effective).[8] When one is subject to this version of confirmation bias, one is more likely to encounter a variety of individually confirming bits of evidence, even if they are undermined by other evidence that is ignored. Then, the More Evidence Heuristic can exacerbate the problem by potentially allowing one to miss the fact that the cherry-picked evidence does not cohere.
Another form of confirmation bias is pseudodiagnosticity, which occurs when one concludes that E confirms H on the basis that P(E|H) is high, without considering whether P(E|H) is higher than P(E|¬H), which is must be if it does confirm H. For example, a murder by stabbing occurs in Amanda’s apartment. Police swab the sink in the apartment and find Amanda’s DNA in it. They consider the fact that if Amanda committed the murder, it is likely she would have left her DNA in the sink when cleaning the blood off her hands, especially if she cut herself. They therefore conclude that this confirms that she committed the murder. However, if she did not commit the murder, her DNA would still be expected to be in her own sink. It is plausibly less likely her DNA would be in the sink if she committed murder since she may have cleaned the sink. The evidence does not confirm that she is the murderer and may confirm her innocence, but by only focusing on one side of the equations one can produce an illusion of confirmation.
Combined with the More Evidence Heuristic, one can start with A and B, which do not actually confirm H to start with, but erroneously conclude that they are confirming evidence through falling victim to pseudodiagnosticity. Then, if one uses the More Evidence Heuristic, one will conclude that there is further confirmation that was never there to begin with.
Combining the More Evidence Heuristic and confirmation bias can lead someone to emphasize evidence for one’s favored hypothesis, at which point this evidence is then assessed using the More Evidence Heuristic without considering how the evidence supports the hypothesis. Suppose one is inclined to accept hypothesis H. As one gathers evidence, one is more likely to notice and remember evidence confirming H. If one nevertheless still gathers some evidence in against H, one can cherry-pick only those bits of evidence that one perceives as confirming H. Then one can evaluate these through the More Evidence Heuristic, and thus reach a conclusion that is radically different from what the evidence supports. The errors compound each other, distorting the evaluation of the evidence to favor a preconceived view.
To summarize, motivated reasoning can lead individuals, especially when supported by a network of like-minded people, to neglect thinking about how evidence fits together to support a hypothesis. They just focus on the number of propositions they have come across that each individually would support the hypothesis, and use the More Evidence Heuristic. This, combined with other biases and errors of reasoning, can lead the More Evidence Heuristic to break down and lead to false conclusions.
(3) What to Do?
While Section 1 demonstrates that the More Evidence Heuristic can be useful, it is clear that we must be on guard against its misuse. There is no way to avoid every possible chance for error, especially given how difficult it is to recognize biases, motivated reasoning, and reasoning errors in oneself. However, there are steps one can take to lessen the risk. The first step is to try to work out the details of how the evidence supports the hypothesis under consideration.
A simple version of this test is just to imagine in detail, a world in which HAB are all true. This is a quick test for coherence, a form of conceivability/plausibility test to test for possibility/probability. Trying to imagine a world where one’s hypothesis and one’s evidence fits together can reveal that it does not fit together after all. This will often help avoid cases such as the Capricorn Club case. Once one thinks about someone going to both Money Stompers and the Blotto Bar, the fact that doing so requires crossing the Tropic of Capricorn and precludes club membership becomes obvious.
One concern with this advice is that it undermines the usefulness of the heuristic. Above I argued that the More Evidence Heuristic is useful because it allows one to reach conclusions without tracking potentially complicated dependencies among propositions. Is the test of imagining HAB in detail asking us to do just that? The answer is no, this is not what the test proposes. The proposed test advises not to work out the dependencies rigorously in detail, but to visualize or imagine how the evidence fits together. This can be achieved without explicitly considering any probabilities or dependencies. It harnesses our intuitive sense of probability in our imagination as a simple way to perform a quick test. In addition, other methods exist to alleviate concerns about misusing the More Evidence Heuristic, as outlined below.
Next, avoidance of closed groups where everyone is on a mission to find evidence in support of one hypothesis can help avoid problems with the More Evidence Heuristic. One will be less likely to be encouraged to focus on number of bits of evidence rather than how the evidence fits together. One will less likely encounter people who have been crafting lists specifically to hide the incoherence of the case for a conclusion behind the number of claims that appear to support it. It will be more likely that the evidence one encounters will be representative.
In addition to the above advice, which is specific to the More Evidence Heuristic, (Wilson, Centerbar, and Brekke 2002) outlines several methods that have been demonstrated to help avoid bias, two of which are applicable to avoiding misuse of the More Evidence Heuristic.
The first applicable method is to compare your conclusions to known biases. If your conclusions co-vary with your biases, this is evidence that bias may be responsible for the conclusion. This can serve as a quick check on reasoning. There is evidence that this is one of the few actually effective methods of avoiding bias. Applied to the More Evidence Heuristic, this means that if you have gathered a great deal of evidence and have concluded that it confirms the hypothesis you wanted to confirm, this is reason to look more carefully at how the evidence fits together. For example, if you know that you don’t want to believe in climate change, concluding that climate change is an environmentalist conspiracy might be evidence that your reasoning results from misapplication of the More Evidence Heuristic.
The other applicable method is to consider the opposite conclusion, i.e. to imagine that the conclusion you reached is wrong and to imagine how that could happen. In short, it is to imagine a world in which ¬H&A&B is true. This has been shown to be effective in revealing blind spots that are not revealed by a simple evaluation of the evidence. It encourages seeing the evidence in a new light. In application to the More Evidence Heuristic, considering the opposite conclusion can encourage one to try to put the evidence together in different ways, which encourages noticing possible incoherence among how the evidence supports the conclusion one reached. To consider how exactly the evidence supports a given conclusion. This can provide an opportunity to notice that perhaps the evidence does not all fit together.
Finally, we need to recognize that sometimes it will not be appropriate to use the heuristic at all. In situations where one has the time and resources to work out dependencies more rigorously and carefully, that will tend to be preferable to using a heuristic. For example, when considering the evidence in favor of a medical treatment, one can perform a meta-analysis on the set of studies investigating that treatment. This is common practice in medical science. Meta-analysis involves careful, (hopefully) rigorous statistical analysis of several studies that have addressed the same question. It does not rely on the More Evidence Heuristic. Rigorous analyses may be required in some cases to be confident that the overall body of evidence provides confirmation. It is common in science that one does not need to resort to the More Evidence Heuristic, because the resources and time to be more careful exist.
The advice here will not completely get rid of the problems associated with reasoning from a growing body of evidence, but they can help to make a useful heuristic safer. With these suggestions in mind, we can better enjoy the benefits of the More Evidence Heuristic without suffering the negative side effects.
References
Darwin, Charles. The Origin of Species. Norton, 1975.
Forster, Malcolm R. “Miraculous Consilience of Quantum Mechanics.” In The Place of Probability in Science, edited by Ellery Eells and James H. Fetzer, 201–228. Netherlands: Springer, 2010.
Gigerenzer, Gerd. “Why Heuristics Work.” Perspectives on Psychological Science 3, no.1 (2008): 20-29.
Gigerenzer, Gerd. and David J. Murray. Cognition as Intuitive Statistics. Hillsdale, NJ: Erlbaum, 1987.
Keeley, Brian L. “Of Conspiracy Theories.” The Journal of Philosophy 96, no. 3 (1999), 109–126.
Nickerson, Raymond S. “Confirmation Bias: A Ubiquitous Phenomenon in Many Guises.” Review of General Psychology 2, no. 2 (1998): 175–220.
Rothstein, Hannah R, Alexander J. Sutton, and Michael Borenstein, eds. Publication Bias: Prevention, Assessment and Adjustments. Hoboken, NJ: Wiley, 2005.
Snyder, Laura J. “Confirmation for a Modest Realism.” Philosophy of Science 72, no. 5 (2005): 839–849.
Thagard, Paul R. “The Best Explanation: Criteria for Theory Choice.” Journal of Philosophy 75, no. 2 (1978): 76–92.
Wilson, Timothy D, David B. Centerbar, and Nancy Brekke. “Mental Contamination and the Debiasing Problem.” In Heuristics and Biases: The Psychology of Intuitive Judgment, edited by Thomas Gilovich, Dale Griffin, and Daniel Kahneman, 185–200. New York: Cambridge University Press, 2002.
Appendix
Proof of Any Combined Confirmation Theorem
Consider arbitrary 〈a, b, c, d〉 ∈ (0,1)×(0,1)×(0,1)×(0,1). I here provide a procedure to construct a probability function P(⋅) such that 〈P(H), P(H|A), P(H|B), P(H|AB)〉 = 〈a, b, c, d〉. To start, we form a partition the space of possibilities with each cell a conjunction of H or ¬H, A or ¬A, and B or ¬B. Let the probabilities of each cell of the partition be as follows:
P(HAB) = s,
P(HA¬B) = t,
P(H¬AB) = u,
P(H¬A¬B) = v,
P(¬HAB) = w,
P(¬HA¬B) = x,
P(¬H¬AB) = y,
P(¬H¬A¬B) = z.
Using the above, the values of relevant probabilities are:
P(A) = s+t+w+x
P(B) = s+u+y+w
P(H) = s+t+u+v (3.1)
P(HA) = s+t
P(HB) = s+u
P(AB) = s+w
P(HAB) = s
P(H|A) = (s+t)/(s+t+x+w) (3.2)
P(H|B) = (s+u)/(s+u+y+w) (3.3)
P(H|AB) = s/(s+w) (3.4)
In order for the probability function P(⋅) to satisfy the conditions of the theorem, we must generate the relevant quadruple 〈a, b, c, d〉 which satisfies all of the following conditions:
s+t+u+v+w+x+y+z = 1 (3.5)
s,t,u,v,w,x,y,z > 0 (3.6)
s+t+u+v = a (3.7)
Conditions 3.5 and 3.6 ensure that P(⋅) is consistent with the axioms of probability. The remaining conditions are to match the target quadruple.
Defining s through z as follows guarantees that P(⋅) is a witness to the existence claim in Theorem 1, where 0 < α ≤ ½ is a parameter:
s = α(1➖a)a(1➖b)b(1➖c)cd (3.11)
t = α(➖a)ab➖s (3.12)
u = α(1➖a)ac➖s (3.13)
v = a➖s➖t➖u (3.14)
x = α(1➖a)a(1➖b)➖w (3.16)
y = α(1➖a)a(1➖b)➖w (3.17)
z = 1➖(s+t+u+v+w+x+y). (3.18)
Proving that a probability function so constructed satisfies all conditions: The definition of z ensures that 3.5 is true.
The definition of v ensures that 3.7 is true.
Regarding 3.8: Substituting for t and x in the left hand side using 3.12 and 3.16, we find 
Regarding 3.9: Substituting for u and y in the left hand side using 3.13 and 3.17, we find 
Regarding 3.10: Substituting for w in the left hand side using 3.15, 
Multiplying numerator and denominator by d/s gives us 
To show that 3.5 is true, we show that each of s through z is greater than 0:
For s: By definition of x, 0 < a,b,c < 1. Thus, s is a product of positive numbers divided by a positive number, which implies that s>0.
For t: From 3.11, s is α(1➖1a)ab multiplied by a factor between 0 and 1 and thus is less than α(1➖a)ab.
From 3.12, we infer that t> 0.
For u: From 3.11, s is α(1➖a)ac multiplied by a factor between 0 and 1 and thus is less than α(1➖a)ac. Thus, from 3.13 we can infer that u > 0.
For v: From 3.11, 3.12 and 3.13, s+t+u = s+α(1➖a)ab➖s+α(1➖a)ac➖s = α(1−a)a(b+c)−s. Now, s is positive, and by definition of a, b, and c, (1➖a) < 1 and b+c < 2. Given that α < 0.5, this implies that s+t+u < a, and thus from 3.14, we conclude v > 0.
For w: Since d < 1, 1➖d is positive. The product of positive numbers is positive, and thus by 3.15 w > 0.
For x: From 3.16 and 3.15, x = α(1➖a)a(1➖b)➖s(1➖d)/d. By 3.11, s(1➖d)/d = α(1➖a)a(1➖b)b(1➖c)c(1➖d), which implies s(1➖d)/d < α(1➖a)a(1➖b). Thus x > 0.
For y: From 3.17 and 3.15, y= α(1➖a)a(1➖c)➖s(1➖d)/d. By 3.11, s(1➖d)/d = α(1➖a)a(1➖b)b(1➖c)c(1➖d), thus s(1➖d)/d < α(1➖a)a(1➖c). Thus, y > 0.
For z: From 3.14, s+t+u+v=a. Since x and y are each equal to w subtracted from (1➖a) multiplied by a factor less than 1/2, x+y < (1➖a)➖2w. Thus, x+y+w < (1−a)−w. So s+t+u+v+w+x+y < a+w+(1➖a)➖w = 1. Thus, z > 0.
All conditions are satisfied. There is a distinct solution for each value of α. The Any Combined Confirmation Theorem is proven.
[1] This paper benefited greatly from comments by the participants in the Tartu Graduate Philosophy Conference in Social Epistemology.
[2] P(first heads|three heads & second heads) = P(first heads|three heads) = 1.
[3] Also see (Forster 2010) for an application of consilience to the success of Quantum Mechanics.
[4] The probability distribution for where it will land in the rectangle is uniform.
[5] (0,1)×(0,1)×(0,1)×(0,1) is the set of all quadruples of real numbers between 0 and 1.
[6] For example, if P(H|A) = 1, then P(H|AB) = 1 as long as the other probabilities are not 0. Also, if P(H|A) = 0, then P(H|AB) = 0. This is why the theorem is restricted as it is.
[7] This describes only a subset of conspiracy theorists and contrarians. There are some who give relatively detailed hypotheses and explain their reasoning in detail. However, many engage in the purely negative reasoning described here.
[8] This is especially a problem in scientific journals, where it is known as publication bias or the file drawer effect. Scientific journals are more likely to publish studies with positive results than negative ones, leading to problems with the overall body of published evidence. The interpretation of published scientific results is difficult and requires care. See (Rothstein, Sutton, and Borenstein 2005) for a thorough discussion of this problem.
Categories: Articles
Leave a Reply