A tour de force on statistics can be found in the ACC Docket, April 2007 at 58. In the context of employment discrimination cases, the authors clearly describe nine advanced statistical calculations and make four recommendations on statistical methodology. Of the calculations, they criticize five of them – chi-square (better to use logistic regression), ANOVA, multiple-measure tests such as the MANCOVA, factor analysis (aka cluster analysis), and stepwise regression – and approve four of them.
The authors favor (1) “protected t-tests like the Duncan range test or Hsu’s test rather than a cascade of independent t-tests; (2) data reliability tests such as Cronbach’s Alpha (but less so the Spearman-Brown Rho and the Kappa and (3) statistical significance criteria of .01 or less. As to the fourth technique, arguments during a lawsuit or in a trial are stronger, “where results are not [statistically] significant,” when the side uses a power test to show that sample size was adequate.”
Turning to methodologies, the authors criticize “mechanistic statistical models that neglect to include social psychological variables such as years of experience, job level, skill, etc.” They urge lawyers and their statistics experts instead to consider using hierarchical linear modeling or multiple regression. Second, they recommend against weighted formulas as well as reliance on personality tests or even discussion of personality traits. Instead, since “personality is not a particularly good predictor of behavior,” it is better to “focus on directly observable behavior.” Finally, the authors advocate that litigants “build a clear set of graphic displays … showing all three pieces of evidence that prove a causal linkage according to Quine’s (1941) classic text.”
If you have any interest in statistics, this sophisticated article is a treasure house of information.