Quantitative approaches to case evaluation have many proponents (See my posts of Oct. 24, 2005: decision-tree risk analysis software; Feb. 8, 2006: a step to prepare for mediation; Jan. 17, 2006: decision analysis; and June 18, 2007: belief nets compared to decision trees.). To estimate the risks of winning and losing and the potential payoffs for different scenarios sounds very buttoned-down, very rational, very progressive. All plausible, but according to the ABA J., Vol. 94, April 2008 at 52, the tools have serious limitations.
A litigation manager in a law department should bear in mind four of the limitations. The article first mentions the sheer complexity of most commercial litigation in federal courts. It cites a case where the decision-tree analysis of an insurance company “set forth 83 different, probability-weighted damage and coverage scenarios.” A second concern is “the tendency to overemphasize those aspects of a problem that can be quantified and to minimize the significance of factors that cannot be easily quantified.” Third, humans too easily discount small probabilities even if the result is massive: a one percent chance of death deserves careful thought. Finally, according to the author, quantitative tools generally do not account well for different attitudes toward risk.
Not surprisingly, numeric case evaluation has its uses and its drawbacks.