Courtroom decisions should not be based on attorneys' manipulations
By Daniel J. Denis
The University of Montana
(THE CONVERSATION) The probabilistic revolution first kicked off in the 1600s, when gamblers realized that estimating the likelihood of an event could give them an edge in games of chance.
Today, statistics has become the dominant way to communicate scientific findings. But courts can be hesitant to incorporate statistical evidence into decisions. Indeed, they have historically been antagonistic toward probabilities and are loath to be swindled by slippery statistics.
However, as an educator of statistics who has consulted in a variety of contexts and has served as expert witness to the U.S. District Court in Montana, I find that both my experience and my review of the evidence suggest that courts increasingly feature statistical thinking whether or not it is identified as such.
Society needs to prioritize educating juries in the language of statistics. Otherwise, juries will be forever at the mercy of convincing, yet potentially invalid, testimony. Courtroom decisions should be based on facts and probabilities, not manipulation by a skilled prosecutor or defense attorney.
-----
Thinking statistically
Probabilities changed the way human beings thought about outcomes. They are a useful tool for expressing our uncertainty about events in the world.
Will it rain today? It will or it will not, that much is certain. But probability allows us to express our ignorance about whether it will rain and quantify the degree to which we are uncertain. Stating "it will probably rain today" constituted a very innovative and different way of thinking.
Probabilities play a role in our daily lives, in decisions from whether to take an umbrella to work to whether to purchase flood insurance. We can consider "statistical thinking" to be any situation where probabilities are involved.
To some extent, humans are intuitive statisticians. For instance, research suggests we can revise a belief in the light of new evidence as prescribed by a statistical theorem, if the probabilities are given in a relatively intuitive rather than abstract fashion.
Statistical reasoning pervades many of the conclusions we draw regarding scientific phenomena. Even physics has had to acknowledge the reality of probabilities. So, if the courts use scientific findings as evidence, probabilities should naturally make their way into courtroom decisions.
-----
Evaluating the evidence
If juries do not understand the nature of statistical conclusions, then they will be tempted to believe that scientific evidence is conclusive and deterministic, rather than probabilistic. For example, probabilities show us that cigarette smoking does not necessarily lead to cancer. Rather, extensive nicotine addiction likely leads to cancer.
Evidence can only fit a theory probabilistically. If we flip a coin 10 times and get 10 heads in a row, that suggests the coin may not be fair, but does not "prove" that it is biased.
Consider the analysis of DNA found at the crime scene. Is the DNA that of the accused? Maybe. Not definitively. A statistician might say, "The probability of this degree of DNA match occurring by chance is extremely small. The match may be due to chance, but since this probability is so small, we may conclude that it likely did not occur by chance, and use it as evidence against the accused."
Of course, human judgment is fickle. Until jurors are trained to make rational decisions based on facts and probabilities, they will continue to be easily swayed by convincing litigators.
In the 1995 trial of OJ Simpson, for example, the bloody gloves found at the crime scene constituted powerful evidence against the accused. The samples obtained were extremely likely to belong to the defendant.
A statistically educated jury would not fall for Johnnie Cochran's classic defense: "If it does not fit, you must acquit." They would know in advance that no evidence, whatever the kind, fits a theory perfectly.
Cochran's statement was, statistically speaking, utter nonsense. Of course no model fits perfectly, but which is the more probable model? That's the task jurors ultimately face, even if they often perceive it as a "guilt" versus "no guilt" decision.
Whenever courts work with DNA matches, they must incorporate acceptable risk and error. But if such uncertainty can be quantified accurately, then it can serve as an aid in decision-making.
Statistical thinking indeed plays a role in the decision between guilt and innocence in a criminal trial. When a jury renders a "guilty" verdict, there is always the chance that the accused is not guilty, but that the many circumstances of the case simply lined up against him or her to lead the jury to a guilty verdict. In other words, the probability of the observed evidence under the assumption of innocence is so low that the evidence likely occurred under a more probable "narrative" that of guilt.
But, when we make such a decision, we do so with a risk of error. This could be quite devastating to a defendant falsely put to death when all along he or she was innocent. For example, when researchers applied DNA testing to death row inmates in Illinois, they found that the results exonerated several inmates.
Errors in probability-based decisions can indeed be costly. Without a grasp of how virtually all decisions are based on probabilistic thinking, no jury can be expected to adequately assess any evidence in a rational way.
-----
Base rates
Courts also struggle with whether and how to use base rates, another type of statistical tool.
A base rate is the probability of some characteristic being present in the population. For instance, say an individual takes a diagnostic test for a disease, such as HIV. The probability that she has the disease would be higher if she were sampled from a high-risk group for example, if she shares needles to support a drug addiction, or engages in promiscuous sex with risky partners.
Courts often ignore base rate information. In Stephens v. State in 1989, the Wyoming Supreme Court heard testimony that "80 to 85 percent of child sexual abuse is committed by a close relative of the child." They ultimately dismissed this, concluding that it was difficult to understand how statistical information would help reach a decision in an individual case.
In another case, a justice of the Minnesota Supreme Court proclaimed that she was "at a loss to understand" how base rates could help predict whether a particular person posed a danger to the public.
Part of the problem is that this information can appear biased against the accused. For instance, consider again the defendant accused of child sexual abuse. The probability that he is guilty might be evaluated in light of the fact that most perpetrators of abuse are relatives of or closely related to the family. This could be interpreted as biasing the evaluation against the accused. However, the courts have considered base rates in employment discrimination cases, an area where perhaps this information seems more naturally relevant (for example, Hazelwood School District v. United States).
If the courts are willing to use base rate information in discrimination cases, they should be encouraged to consider them in other cases as well, even if they seem less intuitive.
Learning to think statistically
Courts should make it a priority to instruct juries on how to interpret probabilistic evidence, so that they are not at the mercy of a convincing, yet potentially misleading, prosecutor or expert witness.
For example, juries might learn elementary statistics through coin-flipping lessons. This could help them, at minimum, find a way to think about the usual "beyond a reasonable doubt" instruction in a criminal trial.
When the assumption of innocence is rejected in favor of guilt, one does so with a risk of being wrong. How much risk is a jury willing to tolerate? Five percent? One percent? Surely such risk must also depend on the severity of the proposed punishment. Every decision is an exercise in risk and cost benefit analysis.
Until juries learn elements of statistical thinking, they are likely to continue making verdict decisions without the appropriate framework in mind. Probabilities have taken over the world, and this fact needs to be recognized by the courts.
Published: Thu, Apr 27, 2017
