The illusion of 'Big Data': Can jury risk be measured?

By Richard Gabriel
The Daily Record Newswire
 
BOSTON — Imagine knowing exactly how a jury would see your case at trial. You’d gaze into your crystal ball, read your tea leaves and the Ouija Board would spell out exactly how jurors would react to your arguments, what they’d do in the deliberation room and what kind of verdict they’d come back with.

We all know that isn’t possible — there are simply too many variables and unknowns in a jury trial to predict what will happen in the deliberation room.

Yet we are enamored with the certainty of numbers. Lately, the legal community has heard the siren song of Big Data: the algorithms used by Google to analyze searches and complex analyses performed on the bar code buying habits of consumers.

Often, when conducting research on verdicts or preparing for trial, we turn to statistics to better calculate risk. We look up previous cases and examine their outcomes. We ask other attorneys about their verdicts. Sometimes, we even conduct statistical jury research to build a jury profile for use in jury selection. Some insurers use computerized programs to analyze and place value on personal injury cases.

However, even armed with such information, incorrect estimations of risk happen more than we may think. A 2008 study found that plaintiffs’ attorneys reject favorable settlements in about 60 percent of cases. Defense attorneys do so less often, in only about 25 percent of cases, but when they do, their mistakes can cost upwards of a million dollars.
While prior case experience and statistical models can provide insight into evaluating a case, the real answers do not lie in the number of verdicts, liability percentages or dollars awarded.  The real ability to evaluate risk lies in qualitative analysis of case factors.

Risk vs. fear and assumption
What do we mean when we talk about trial risk? The number of plaintiffs’ or defense verdicts? The average award? Or are we afraid of how a jury will view a key witness’s testimony, a venue’s reputation for verdicts or a plaintiffs’ attorney’s track record? These may be risks, but sometimes we make generalized assumptions about evidence or non-legal factors without really analyzing how a jury will realistically convert those factors into a liability finding or damages.

We like numbers because they are concrete and they seem more absolute. In a profession with so much unpredictability and the seeming randomness of a judge’s or jury’s decision, having hard figures to reference can be comforting. If the last 10 personal injury cases you and your colleagues have defended have all settled for between $80,000 and $220,000, you might reasonably assume that an upcoming case with similar injuries will be valued in the same ballpark.

But subtle differences in the atmosphere of a case, including the venue, jury composition, the style of the attorney, the personality of the witnesses, word choice in a document, pictures of the incident in question, a judge’s attitude, all can skew the predictive assumptions about its value. They can also change the values in a case drastically, because case value in a jury trial is all based on the perception of value by that jury.

The jury is the equation
While qualitative factors may seem harder to measure, they can be evaluated and systematically scored just like a prior jury’s verdict or an expert’s report on a life care plan. The key to a more accurate risk-prediction, then, is to not just look at hard numbers, averages, and statistics, but to closely examine the unique issues in your case in relation to the data you’ve already gathered. For example, let’s take a risk scale of between 1 (low risk) and 5 (high risk). You might then divide your case into several categories: case facts, legal rulings, exhibits, testimony and non-legal issues.

Obviously, if you are defending a hospital in a medical malpractice case and your client has lost the patient’s chart, you might assign a risk factor of 5. Similarly, if you represent the plaintiff in an employment case and a judge has ruled that your client’s past drug use can be admitted as evidence, you might give that a 4 or 5 risk factor, depending on the relevance to the employee’s termination. A judge’s ruling on allowing punitive damages significantly increases the risk factor for a defendant.  But what about the non-legal elements of a trial? How would we rate them?

We often speak about how sympathetic an injured plaintiff will be to a jury. However, jurors can be sympathetic and still vote against your client. To evaluate this risk score, it is better to look at how much a jury will want to help the plaintiff.

When it comes to damages, it is always important to ask how angry a jury might get when they hear certain testimony or evidence. The angrier an issue might make them, the higher the risk score. You also may be able to estimate how much you will be able to sooth that anger or reduce the risk score through other evidence or re-direct examination. Other risk factors may include how consistent or inconsistent they perceive arguments to be or how caring or arrogant they perceive the witnesses in the case to be.

Anaheim, Calif. has recently had officer-involved shootings, which have triggered protests and editorials about the deterioration of city services because of resources allocated to the resorts around Disneyland. Criminal cases, excessive force cases and even business disputes involving hotels in Orange County have higher risk factors in this environment.
These factors are more meaningful predictors of risk because they absolutely translate into jury apportionment of liability, percentage of fault and damages. They are the motivation behind the desire to check a certain verdict box or put a dollar figure on the worth of a contract or business.

Don’t become another statistic
Once you have identified all of the issues that you think will impact how the jury sees your case, you will want to calculate a final score. The point of that analysis is to evaluate the “jury risk” in the case — how a jury is likely to evaluate the issues from a layperson’s point of view. To do this, you can conduct focus groups or mock trials, gather a collection of friends, or run your case by a cab driver or server in a restaurant. Whatever you choose, it is crucial to look for all potential risk, even if it registers only a 1 or 2 on the risk scale. It is also important not to dismiss or minimize the risk by saying, “A judge would never let that in,” or “A jury will never buy that.” Once you have compiled your risk categories and issues in each category, you can tabulate a more meaningful risk score. Once you have your total score, you can also weight it properly. One issue, by itself, may significantly outweigh all of the other issues. Additionally, if you have the first jury trial in a series of pharmaceutical cases or an MDL, the risk factors may need to be weighted more heavily.

This exercise is not intended to make you doubt your case or lose confidence in your abilities as an advocate. It is intended to provide you with a richer appreciation of how jurors make decisions and give you an opportunity to systematically analyze the jury’s interpretation of the issues as the basis for risk evaluation.

While we’ll never be able to predict the outcome of a trial with 100 percent accuracy, conducting a qualitative analysis of your case and its risks will always put you in a better position than if you rely on hard numbers and quantitative data alone. Understanding what issues trigger positive and negative reactions in jurors will help you get a better understanding of what the true risks are in your case, and provide insight on how to tailor your trial presentation to any jury if the case goes to trial.

—————

Richard Gabriel is the president of Decision Analysis, a national trial consulting company, and co-author of Jury Selection: Strategy and Science.