Robert Half Legal reports data in ACC Docket, Vol. 30, June 2008 at 52, that come from a “survey of 150 lawyers from among the largest corporations in the United States and Canada.” From the standpoint of reliability, are these lawyers of relatively similar seniority? How were the lawyers selected? What percentage of the lawyers who were invited took part?
Overall methodology aside, I really wonder about the results from one question: “Of the following, which would you say is your legal department’s single greatest business challenge today?”
Increased workloads 34%
Compliance or regulation issues 28%
Controlling litigation or outside counsel costs 24%
Budget restrictions 7%
Employment issues (e.g., hire, retain, morale) 6%
Don’t know 1%
Multiple-choice questions are problematic because if you omit choices that are important, your findings will be under a cloud (See my post of Dec. 3, 2007: tricky aspects of multiple-choice questions.). For example, some lawyers might feel that the biggest challenge they face is the growing complexity of their legal issues, management of knowledge, lack of teamwork, unreasonable clients, or ineptness of technology. None of those choices were available so do the results have much meaning?
More fundamentally, the choices available overlap. “Increased workloads” could mean employment issues such as hiring constraints or they could evidence an inability or unwillingness to hire outside counsel (See my post of May 21, 2008: “workload/time pressure” as top choice of an Australia/New Zealand survey.). “Compliance or regulation issues” presumably contribute to increased workloads. “Budget restrictions” make it more difficult to handle increased workloads and compliance/regulation issues. Beset by methodological and logical weaknesses, the metrics have little value.