ASA issues guidance on statistical statements for forensic evidence

In response to concerns that the use of forensic evidence such as shoe prints, fingerprints, bite marks, fibers or hairs has contributed to wrongful convictions, the American Statistical Association (ASA) has released a document with guidelines for discussing forensic evidence.

“Past errors provide us with opportunities to work with the forensic science community to improve the process and strengthen the use of forensic evidence,” said ASA president Karen Kafadar. “This statement is just one step in that direction, but we hope it will be an important step.”

In particular, the ASA wants experts to base their testimonies on databases and quantitative analyses, rather than subjective personal experiences.

“The forensic practitioners are dedicated and well-meaning—they see agreement between a crime scene sample and one provided by a suspect and they’re trying to assess the relevance or value of that evidence for prosecuting the crime,” said Hal Stern, vice-chair of the ASA Advisory Committee on Forensic Science. “One point of the ASA statement is that if you’re overstating or understating the value of the evidence, there are several harms that can occur. If you overstate the evidence, you might put the wrong person in jail, but just as importantly, the guilty party would go free.”

As one example of the relevance of the ASA statement, consider a case in which a crime scene shoe print is found. Forensic practitioners analyzing the print will identify features of the print (e.g., size, model, tread pattern, wear pattern) and determine if they match the corresponding features of the suspect’s shoe. The ASA statement makes clear that practitioners should use data to assess the probative value of these matching features. “How many shoes would have this tread pattern; how many shoes would have wear of this type?” Stern asked. “Answers to such questions would allow a forensic practitioner to opine about the strength of this particular evidence.”

In this example, to align with ASA recommendations, an expert might identify the corresponding features, and then testify based on data about the likelihood of matching size, model, tread pattern and wear pattern if the suspect did not commit the crime (i.e., the likelihood that another person from the population would have matching features).

New technologies changed how the justice system viewed forensic evidence, said Matt Redle, former prosecuting attorney of Sheridan County, Wyoming, who served on the National Commission on Forensic Science (NCFS). The commission, jointly run by the National Institute of Standards and Technology (NIST) and the Department of Justice, existed from 2013 to 2017 and made recommendations to the Department of Justice to enhance the practice and improve the reliability of forensic science.

“We really didn’t know a lot about this, and we assumed the forensic evidence we were working with was true and correct,” Redle said. “Frankly, what was the eye-opener for prosecutors and the system writ large was the information we received from DNA and the education we got about what good evidence looks like.”

Better practices with forensic evidence will ultimately benefit the criminal justice system, Redle said, even if the new principles for practice make it more difficult to admit certain types of evidence in the short run.

“Credibility in the system is the greatest asset prosecutors have,” Redle said. “There’s damage we do to the system and credibility in our rule of law if we’re hiding the ball from our juries about what the limitations [of forensic evidence] are. It’s far better to be open and upfront and recognize the limitations.”

The ASA statement extends a document drafted but not passed by the NCFS. The ASA Advisory Committee on Forensic Science, under the leadership of Kafadar and Stern, updated that document with the latest statistical research.

“Ultimately, we hope this statement will be useful in explaining to stakeholders—forensic scientists, law enforcement officers and courtroom personnel—the data necessary for making assessments about sources of evidence,” Kafadar said. “We also hope the document will guide forensic scientists toward the type of research needed to strengthen and quantify the value of evidence being presented in criminal cases and thereby increase our confidence in the judicial system.”

The ASA statement encourages practitioners to follow “reliable and valid” scientific processes, audited by an independent science organization such as NIST, as the NCFS recommended.

“The ASA statement being adopted by the entire organization is extremely important because the ASA has an extraordinary reputation and a very large membership,” said Peter Neufeld, co-founder of the Innocence Project, the nonprofit legal organization that exonerates the wrongly convicted through DNA testing and reforms the criminal justice system to prevent future injustice. “We have plenty of first-hand encounters where forensic examiners relied on their subjective impressions and personal experience instead of statistical data to explain the value of the evidence.” Neufeld continued, “As a consequence, there were serious miscarriages of justice, where factually innocent people lost decades of their lives and some of them were sentenced to death. Hopefully policymakers, judges and people who really care about the strength of evidence used to make decisions about life and liberty will take notice and do everything they can to apply rigorous scientific and statistical principles in the future.”