Articles Posted in Benchmarks

A survey this blog addressed recently (See my post of Oct. 30, 2006 on an insurance industry study.) gave law department lawyers ten attributes to evaluate. Rather than ask them to “check all that apply” (See my post of Dec. 20, 2005 that castigates this methodology.) but not going so far as to ask them to allocate 100 points among the attributes (See my post of Aug. 14, 2005 that praises this methodology.), the survey chose a third method. It asked respondents to pick the top three from the ten. To ask respondents to pick their top two, three or four of a list vastly improves on the check-all-that-apply technique even if it falls short of – but is easier than – the allocation method.

What bewildered me was that only 88 percent of the respondents ranked in their top three attributes outside counsel’s expertise. Amazing, isn’t it, that 12 percent of the respondents did not see expertise of the law firms they chose as ranking higher than fourth on their personal listing! Perhaps some people answered the survey as if they were taking for granted the capability of law firms, so they focused on what other factors differentiate them. Perhaps “expertise” is too exalted a term for some, a rarely achieved or needed level of extraordinary competence that is not often an attribute that departments need in a firm.

Once more I attack the Nov. 10, 2005 report by The Corporate Legal Standard, and specifically one item in a table entitled Top 25 Law Department Operations Metrics (at 10). One mouthful of a metric sounds innocuous yet impressive: “ratio of law department business processes undergoing automation/business processing reengineering/Six Sigma — TQM — other quality improvement.” But pause to deconstruct it.

The subjectivity of deciding when a law department has undertaken an initiative to improve the quality of any of its “business processes” detracts considerably from the reliability of this metric. Even if you were able to pinpoint the start of each initiative, there is no accepted taxonomy of law department “processes” (See my post of Oct. 18, 2006 on processes.). This point leads to another criticism. The law department would need to catalog all its processes to calculate the ratio of those undergoing quality improvement efforts.

Those of us who care about law department management ought to have our “reach exceed our grasp, or what’s a heaven for?” But the firmament is not big enough to hold this ethereal metric.

The Nov. 10, 2005 report by The Corporate Legal Standard includes a table entitled Top 25 Law Department Operations Metrics (at 10). Most of the proposed metrics are quite standard; two of them I would like to single out (See my posts of Sept. 13, 2006 for other comments on this report.).

One dubious metric is “total liability of matters handled by the law department.” If “liability” is defined as the amounts sought by plaintiffs in lawsuits or ADR proceedings, one could add up all the monetary claims. Such damages sought by plaintiffs, we all know, can be grossly inflated figures — and the vagaries of punitive damage assessments push them even higher. Hence, even in litigation the “total liability” metric has no quantification or real-world significance.

If “liability” covers even more broadly the risk of damages arising from a contract breach or dispute, the foolhardy law department that tries to limn that exposure would become a laughingstock. At its broadest reach, “liability” potentially lurks behind everything a law department touches: advice on environmental remediation, counsel on employment matters, due diligence in acquisitions, and so on forever. Total liability is a wrong-headed metric.

Chicopee, MA Mayor Michael D. Bissonnette was quoted in The Republican, Oct. 27, 2006 on development of the city’s properties. Evidently the city has had no central register of its properties such as schools, parks, strip islands, and right-of-ways.

The Mayor said he is analyzing a report completed by a Law Department intern of every piece of property the city owns; but not just a report — one that weighs in at 700 (sic) pages.

Is it the fact that the Law Department has an intern that awes me (See my posts of March 25, 2005 April 23, 2006 on interns.)? Is it that modest-sized Chicopee has a Law Department? Or that this work shouts out quasi-legal?

Every few years, early in a benchmarking project, a general counsel asks whether sharing data on staffing, spending, cases pending and the like risks anti-trust scrutiny. One law department I know went so far as to obtain an opinion of outside counsel that the benchmark study would not jeopardize the company. Is the law department’s company at risk if it contributes its data to an industry benchmarking project?

Without expressing a legal opinion, and with no intention of rendering legal advice in any jurisdiction, I think benchmark projects trigger no antitrust wrongs. No one conspires in restraint of trade – except perhaps to pay less to law firms – and no one fixes prices, although they may want to find ways to pay less to law firms.

Laurence Simon’s Guide To In-House Counsel Salaries (2006) states that “around 600 of the [Irish] Republic’s 6,500 solicitors currently work in-house.” Elsewhere I have noted the ratio of in-house counsel to outside lawyers in England (See my post of April 13, 2006 with its estimate of 14.5%; and Feb. 1, 2006 No. 3.). I have also seen estimates that approximately 100,000 lawyers in the US work in private corporations or the government, out of a total of around one million lawyers. Observers estimate that there may be 50,000 in-house counsel in China, but I do not know how many total lawyers (See my post of Oct. 19, 2005.)

Based only on the first three data points, a rough rule of thumb might be about one in-house lawyer for every 10 private practice lawyers. That’s not at all to say that in-house lawyers each sustain ten outside counsel; quite differently, the typical 40/60 inside to outside ratio of spending along with the higher hourly cost of outside counsel suggests more of a one-to-one ratio (See my posts of June 28, 2005 on the 40/60 ratio; March 19, 2006 and its application to Canadian data; and Aug. 14, 2006 on the one-to-one derivation.).

Previously I have explored ways to depict and analyze law department data (See my two posts of Oct. 1, 2006 on graphics and visual techniques.). Other techniques of data exegesis are available for law department managers.

One statistical tool is the “Gini coefficient”, a measure of concentration. If the Gini coefficient were calculated for the concentration of litigation, a score of zero would mean all companies within the industry have the same number of lawsuits, and a score of one would mean a single company has all the lawsuits.

A fascinating study looked at 20 years of law suit data for about 2,000 of the largest U.S. corporations (Dunworth, Terence and Rogers, Joel, “Corporations in Court: Big Business Litigation in U.S. Federal Courts, 1971-1991,” 21 Law & Soc. Inq. 497, 531 (No. 3, Summer 1996). Within industries, the level of lawsuit inequality—one or a few companies facing the bulk of the litigation—is extremely high. For example, in Transportation/Communication, far more of the lawsuits tracked by this massive study befell fewer companies (Gini coefficient of .78) than in Food/Agriculture (a Gini coefficient of .52). This data has application in benchmarking projects and in self-assessments by law departments.

A fellow consultant claimed recently that a law department has the right to obtain from its law firms cost data on closed matters, even when those matters were other clients’. The data so gotten will help the law department establish typical costs incurred by types of matters.

A benefit to a law department does not justify a wrong to a firm. One wrong is that different clients invoke different strategies in litigation, so their costs vary for reason other than the effectiveness of the law firm. A second wrong is that the attorney-client privilege might cloak billing information. To learn enough about a set of matters to enable the data to make sense might breach the privilege. Third, there is no taxonomy that lets a firm decide whether a given lawsuit is similar enough to the archetype to include it in a data set. Finally, how would the requesting law department feel if its matter data were included in a database shipped to a competitor?

Publicly-traded companies in the US must disclose each year the compensation of their five highest-paid executives. What those executives make in base, bonus, stock options, restricted stock grants and otherwise is all laid out in tables. It’s fairly easy to use that data, when compiled for general counsel in an industry along with who belongs in the top five, to estimate the compensation of a general counsel who is in the industry but not in that quintet.

As an example, for the general counsel of a bank I took the publicly-available data from those banks whose top legal officer was in the five-most highly compensated group. From the data on CEO base pay and the other executives’ base pay, I could calculate trend lines. Since we knew my client’s top executives’ compensation, we could come close to what his base pay should be in light of the medians of the other available data.

Several previous posts have explained techniques to make sense out of data (See my posts of April 5, 2005 and May 10, 2005 on correlation; Nov. 30, 2005 on averages, medians, and modes; Aug. 14, 2005 on multiple regression; May 31, 2005 on bell curves; Nov. 13, 2005 on power law distributions; March 10, 2005 on the difference between linear and exponential growth; and of Jan. 26, 2006 on Bayesian statistics.).

Another methodology to explain metrics, with ample examples, applies the techniques of visual depiction.

Column charts, the picket fences, such as spending year over year, make immediate sense to everyone (See my post of Nov. 19, 2005 on Google showing its legal staff growth rate over a few years.).