Epistemic value of multiple eyewitness accounts: single event vs. multiple events given a fixed number of eyewitnesses?

Question

Intuitively speaking, multiple independent eyewitness accounts of a single event are more convincing than a single eyewitness account. For example, multiple independent eyewitness accounts of a loud explosion in a remote area (e.g. from different locations and viewpoints) are more convincing/reliable than a single account (e.g. maybe the single witness hallucinated the explosion).

A bit more formally, if we define X as some truth claim about some event, process or phenomenon in the real world, we could say that:

P(X is true | multiple eyewitness accounts) > P(X is true | a single eyewitness account)

However, what happens if we keep the number of eyewitness accounts constant and only change the number of events?

For example, let X = "alien abductions are real", and let E_i be a concrete example of an (alleged) alien abduction. X is a general claim, E_i is a claim about a very specific instance of X. It is clear that E_i entails X. Thus, which of the following probabilities is greater than the others?

• P(E₁ is true | N eyewitness accounts for E₁)
• P(E₁ is true or E₂ is true or ... or E_N is true | one eyewitness account for E_i, for each i in {1, ..., N})
• P(E₁ is true or E₂ is true or ... or E_N/2 is true | two eyewitness accounts for E_i, for each i in {1, ..., N/2})
• P(E₁ is true or E₂ is true or ... or E_N/3 is true | three eyewitness accounts for E_i, for each i in {1, ..., N/3})
• Etc.

In other words, given N eyewitness accounts, what is the optimal distribution of eyewitnesses over specific alleged instances of X that maximizes the probability of X being true? What should be more convincing, 1000 eyewitness accounts for E₁, 500 eyewitness accounts for E₁ + 500 eyewitness accounts for E₂, etc.?

Note: I used eyewitness reports of abductions by aliens as an illustrative example, but the reasoning can be extended to other rare events, such as reports of miracles, paranormal phenomena, angelic encounters, Bigfoot sightings, testimonials from whistleblowers (conspiracy theories), etc.

---

Related questions

• How to model the probability of a truth claim given an arrangement of eyewitness accounts supporting specific instances of that claim?
• Bayesian argument for combining extraordinary claims
• Backwards Bayesian argument for alien visitation?

Answer 1

I will defend the following 4 contentions. I do not claim that they follow from natural law or mathematical proof--they are merely designed to represent actual human behavior:

1. All else equal, multiple witnesses reporting one event each is more compelling than one witness reporting multiple events
2. Multiple independent witnesses are more compelling than the same number of dependent witnesses
3. Increasing the number of events/witnesses will eventually experience diminishing returns
4. In the real world, "all else" is not equal, and one credible witness is more compelling than multiple less-credible witnesses

--

1. One witness or many: Whistleblowers

The OP cited whistleblowers, which are an effective example to defend contention one. It is common practice in a corporate environment for there to be a small subset of employees who are angry, loud, and dissatisfied. Corporate surveys tend to discount these voices if they are numerically overwhelmed by different perspectives.

In this case, one disgruntled employee who voices numerous concerns is going to be taken much less seriously than multiple employees voicing similar concerns.

Evidence of this can be found in examples such as:

• Organizational Health Indices, which aggregate large data sets and use averages to compare company performance with peers. In the mathematical summaries of these surveys, one very angry, long response with many examples will be weighted no differently than one very short, satisfied response. One dissatisfied voice will do little to drag down the company's score. Multiple dissatisfied voices--especially if they are dissatisfied for similar reasons--will make a bigger splash.
• Voting: most political organizations with democratic norms acknowledge that a better result is achieved if each person votes once than if some people vote more than once. So strongly do most nations believe this, that in most jurisdictions, voting more than once is a crime that comes with hefty penalties. Lots of people voting once rather than one person voting many times is an example of placing trust in multiple attestation.

As a result, lone whistleblowers often take significant blowback, even if they are reporting serious violations in good faith--this is why whistleblower protection laws exist.

--

2. How multiple attestation works for historians

Most events in human history are not preserved in any extant records. And most events that are preserved survive in only one source. For this reason, especially in the study of ancient history, having two sources for an event is a very high, very important standard, which historians refer to as the criterion of multiple attestation.

However, if we have two sources and source 2 is quoting source 1, this does not count as multiple attestation. We could just as well write another work today quoting source 1 and claim we now have triple attestation. Source 2 is only effective in corroborating source 1 if there's reason to believe source 2 isn't simply regurgitating what source 1 said.

The most prominent usage of the criterion of multiple attestation is seen in New Testament studies, and particularly in relation to the Synoptic Problem (disclaimer, the linked source is my own work on the Synoptic Problem). If one Gospel appears to be quoting another, this is single attestation. If two Gospels have the same event recorded independently, this is multiple attestation (what fits in which category depends very much on how one solves the Synoptic Problem).

To give one, concrete example from Roman history: Dio Cassius, Orosius, and the Acts of the Apostles all tell us that Emperor Claudius made a decree expelling the Jews from Rome (circa AD 49). Each provides distinct details that could not have been obtained from the other two. From the standpoint of a historian of ancient history, this is the gold standard (we'll call finding an inscription the "platinum standard" ⁼⁾) - this is a very solid piece of history and the likelihood is much, much greater that the decree happened than that it didn't.

--

3. Diminishing returns: We've heard this story before

Alien abduction is a good example here. There have been so many stories of alien abductions told that many are numb to it. Hearing one more alien abduction story is very unlikely to persuade someone who has already heard 100 of them and didn't believe any of them.

Additionally, as pointed out in other answers, because so many alien abduction stories have been told, people can be influenced by each other's stories, diluting the value of an incremental alien abduction story.

An illustrative example is the JFK assassination (arguably the grand-daddy of all conspiracy theories). For those who believe the lone gunman theory, hearing yet another alleged eyewitness account that proves a conspiracy, or another complex cover-up scheme described, can be exasperating--it's not going to move the epistemological needle. Ironically, even if many of the witnesses & theorists are wrong, it does not necessarily mean all of them are wrong. In this sense, having too many witnesses can be damaging: if 10 people claim they saw a man on the grassy knoll shoot JFK, and 9 of them were mistaken...JFK was still shot by a man on the grassy knoll--but the erroneous witnesses make it the claim less credible, not more.

--

4. Credible Witnesses

In both the ancient history and the JFK examples, the credibility of a witness can overpower (positively or negatively) the quantity of witnesses.

If most of recorded history is preserved by a single source, do we discard most of what we think we know about history? No. We evaluate the credibility of the historian by comparison to other sources of information. If the historian gets it right when we can fact check the claim, we have a greater level of confidence that the historian is correct when we cannot fact check the claim.

In practice, one highly credible witness can provide a compelling basis to believe something even if it is opposed by numerous, less-credible witnesses. Because people tend to consider themselves to be credible witnesses, we have sayings such as "I saw it with my own eyes", where despite receiving criticism for doing so, people believe something others disbelieve, because it is consistent with their own experience (since all evidence arises from experiential evidence, this practice is not wholly irrational).

I'll offer one specific example here as well: what was the last song played by the band of the Titanic? (spoiler alert, we're not going to solve that vexed question here).

This question is interesting and relevant, because most Titanic survivors who spoke on the matter said the band played Nearer My God to Thee, and for many years this was uncritically accepted as correct. However, more recent historians have challenged this claim, largely on the basis of a single witness: Harold Bride.

Bride was the junior wireless operator on board, his profession was one which required attention to detail and a sharp memory, and he was closer to the Titanic in its final moments than most other survivors were (he survived aboard the overturned collapsible B lifeboat (ibid)).

Bride said nothing about the band playing hymns, but spoke of their playing lively tunes.

The Titanic musical debate rages on with no end in sight, but it is a useful example of my 4th contention. Many are prepared to take the word of one, particularly strong witness over and against the majority of witnesses who said something different.

--

Conclusion

Eyewitness testimony cannot be ruled out entirely, as it is the ultimate basis for all data that we have to work with. We look for aggregation and repeatability in order to establish credibility.

On a matter as subjective as judging the accuracy of an eyewitness, we should not expect to find a formula for weighing one claim against another. But in actual practice, all else equal people tend to take more seriously something they hear from multiple sources versus than something they hear repeatedly from one source, but credibility of sources can readily outweigh quantity of sources.

I submit that the most secure epistemological claims are those that come not from quantity but quality of sources.

---

---

^{Which sources are high quality? This requires epistemological evaluation as well...}

^...loop...

^{...crash! =)}

Answer 2

Epistemic value of multiple eyewitness accounts

I don’t know if this is an answer, but here is an example from real life: Babe Ruth’s called shot from the 1932 World Series. Look it up at Wikipedia > Babe Ruth’s called shot.

It is a great story, and relevant here. In the fifth inning of game three of the series, Babe Ruth was behind the pitcher, Charlie Root, with two strikes against him. Ruth then made some pointing gesture which is still debated today. Ruth then hit the next pitch for a home run near the center field flag pole. Did the mighty Babe call the home run before the ball left Root’s hand?

Then followed 80 years of controversy. The eye witnesses, all 30,000 of them, say that Ruth was calling the next pitch. Ruth himself said different things. Apparently, surviving newsreels of the event do not resolve the issue.

The relevance here is that the event and its controversy show the difficulty of the question. Do the called shot supporters prevail just because there were 30,000 of them?

Answer 3

The experience of science is that eyewitness accounts are very low value items in general. That is to say, there is no such optimal distribution as you suggest. Events reported purely through eyewitnesses are going to remain entirely suspect no matter how many times they are repeated or what distribution of persons viewed them each time. It's "The Easter Bunny" all the way down.

Consider, for example, witnesses to E2 who have previously become aware of reports of E1. This could well convince us that their reports are far less credible. Reports of aliens are in the news and popular entertainment. If somebody stumbles in from a corn field reporting little gray people did things to his buttocks, we naturally attempt to perform some type of field sobriety test. And we will do this no matter how many times such occcurs.

What is required is careful eyewitness accounts, careful vetting of the witnesses, and some form of corroboration. Here are just a few.

• A witness who has passed that sobriety test is more credible than one not so tested.
• A witness who has some other type of evidence besides only his claims is more credible.
• A witness who is unaware of, unknown to, and unrelated to a previous events or witnesses is more credible. Though this lack of contact is naturally difficult to be sure of.
• Predictions of future occurences that are then confirmed are more credible than tales of aliens "who got away." Particularly if the predicted event is observed by "hostile" observers who are attempting to disprove the hypothesis.
• Physical evidence of some sort can be interogated in ways that a witness cannot. This can come in a large number of forms from photographs, to impact sites, to material, to an actual member of the Asgard Fleet (Stargate reference) coming down to say hello.

Each of these is made more credible by being examined for evidence of self-delusion or fraud and being found innocent. There are a large number of such examination methods from reproducibility to double-blind to looking at the ID info on a photograph, etc.

Answer 4

Epistemology doesn't use concrete mathematics, so you're unlikely to find a distribution that maximizes the probability of X being true (and if you do, I wouldn't be quick to trust it).

Generally speaking, you want at least enough eyewitness accounts for at least one event so you can say that event is true independent of the other events. How many eyewitnesses there are also isn't the only/most important factor, but rather the sum of those eyewitness accounts should be reliable and consistent (with one another, with reality as we know it and with any other evidence we may have for the event), so that the event being true becomes the most likely explanation for those eyewitness accounts.

Beyond that, it doesn't seem like it would make a big difference whether additional eyewitness accounts are for the same event or for other events.

Multiple events that can all be concluded to be true independently is possibly slightly better than a single event, as this is more difficult to explain with e.g. a mass hallucination. Although multiple events spread over a long period of time may be very susceptible to eyewitness accounts being influenced by one another (one person claims to have been abducted by aliens, other people hear about that and that makes them more likely to attribute some experiences they have to abduction), which reduces their reliability greatly.

Alternatively, even the extreme of one eyewitness per event can be trusted if you can compare and corroborate the eyewitness accounts with one another in a way that can't reasonably be explained through other means (e.g. if eyewitnesses give detailed descriptions of the aliens that abducted them, but these match the appearance of aliens in popular media, that would make their accounts unreliable).

---

Note: there is an important distinction between what people experienced (as in the raw signals their senses send to their conscious brain) and how they interpret that. We could generally say that what people claim to have experienced is accurate (except in cases of (a) misremembering, which is quite common, and (b) intentional lies, which is less common), while we reject their interpretation of that experience. If someone claims to have seen Bigfoot, they may in fact just have vaguely seen something moving in the woods (which we may accept as true), and attributed this to Bigfoot (which we reject). It may not always be easy nor possible to distinguish the "raw" experience with the interpretation, and the line may be blurry, but we should try our best to distinguish them.

We should take their interpretation into account, but we should focus much more on their "raw" experience, because interpretations can trivially be wrong, while "raw" experiences tend to be more trustworthy (although they are far from fully trustworthy, as people can hallucinate, dream, misremember, etc.). Depending on the "raw" experience, we can try to determine how likely their interpretation is to be correct (compared to other possible interpretations).

So it's not strictly a question of "we have N witnesses confirming event A", but rather "we have N witnesses claiming to have experienced B, which they attribute to event A", and then we evaluate whether event A is the best / simplest explanation for those eyewitness accounts (while also considering the possibility of lies or false memories).

Answer 5

Ok first of all your N is limited by having a finite number of people who are physically able to observe the event, yet you may have more "eyewitnesses" coming out of the woodwork to give their account. Which raises the question of "How credible are eye witnesses". And the answer is not very much. Like they give a biased account that is on it's own not reproducible and that might not even stay consistent if prompted for multiple times.

Also how do you define "independent"? Like as soon as a group of strangers is close enough to each other to be aware of the other strangers existence they are somewhat no longer independent as they'd have more confidence in their account because they weren't the only one who saw it. Not to mention if they had time to talk to each other and sync their accounts. Or hear other people's testimonies before their own.

Which severely limits the possible N, before an increase in N does not result in an increase in credibility. Because of EVERYBODY KNOWS then you'll have "self-correcting" effects on the accounts, rendering them useless despite the raw number.

Also again, those accounts are biased so just because you have a large number of events or a large number of witnesses per event or ideally even both, that does not mean that they've actually seen what they say they've seen. Like they've probably seen something but their interpretation might be way off from the truth. Like take a stage magician performing an illusion and ask the non-initiated audience what happened and despite having lots of eye witnesses the accounts might be completely useless in figuring out the trick even if they align. Because unless they are likely falling for the same distractions and then describe it as magic.

So what you get is not actually P(E_i|...) but just P(E|...), so just the probability of an event and E₁, E₂ and so on can be independent and just linked to each other because of a bias to expect a sequence.