Articles Posted in Benchmarks

General counsel deserve to have good, reliable, and useful benchmark figures. Many people try to supply that need. All of them face some systemic unknowns about the representativeness of the law departments that participate in their surveys.

Centralized departments: For example, data may be less common from decentralized legal functions as well as less complete (See my post of Feb. 16, 2011: perhaps not full staff or spending figures.).

Sizable departments: Participants may tend toward the larger size since they are pursued more (See my post of July 31, 2011: possible bias toward size.).

Return on investment, the calculations that justify spending money, figure prominently in decisions by law department managers – or they ought to. Being so important, many posts on this blog aim directly at ROI or refer to it and it has become time for me to invest in updating my previous collection (See my post of Oct. 22, 2008: ROI with 17 references.).

Some of the subsequent posts refer to the calculations themselves (See my post of Sept. 1, 2009: allow for the time-value of money; Sept. 2, 2009: calculating ROI benefits and costs; Sept. 2, 2009: part two of calculating ROI benefits and costs; Oct. 19, 2009: how to figure the ROI of CLE; Nov. 13, 2009: ROI calculation for an entire legal department, the holy grail; and May 3, 2010: vendors and preparation of ROI document.).

More of the recent posts refer to returns on investment of specific types of initiatives (See my post of Feb. 6, 2009: inventor rewards; April 8, 2009: collaboration; April 27, 2009: videoconferencing; June 4, 2009: e-billing; Aug. 25, 2009: technology projects; Nov. 5, 2009: matter management systems; Jan. 25, 2010: AFAs, especially fixed fees; Jan. 25, 2010: administrators; March 1, 2010: recruitment ad; March 29, 2010: matter management system; and July 23, 2010: compliance investments.).

Metrics and benchmarks provide opportunities for sleight of hand, for tricks that fool some people, for willful ambiguities, and for dumb decisions. Here are eight candidates for booby prizes other than samples that are not representative and questions that are not neutral.

Frame the findings with “save up to” or “reduce up to” language (See my post of June 2, 2011: misleading to tout extremes.).

Use indefinite numbers like “lots of” (See my post of May 24, 2011: Kraft’s law department clarifies estimative language.).

People in legal departments who want to make sense out of the large volume of data generated by their various software programs, notably their matter management system, have several choices for how to dig through and make sense of that data.

Open source software called Hadoop lets users sift through massive data sets quickly and cheaply. Two startups, Datameer and Cloudera, offer variations that simplify use of Hadoop. Software like this may liberate law departments from thralldom to rows and columns as their architecture of storage and search; according to Bloomberg BusinessWeek, Sept. 12, 2011 at 72, “Hadoop can handle data that don’t fit into spreadsheets.”

This topic is beyond me technologically, but it suggests much more powerful, yet relatively inexpensive tools to help us learn from the increasing volumes of data law departments accumulate. Here are several related posts (See my post of Dec. 18, 2006: five common ways to report on data; Dec.12, 2007: Pfizer and Business Objects for its dashboard; May 8, 2008: exporting data from a matter management system to a spreadsheet; Feb. 15, 2009: third-party report writers; March 20, 2009 #1: visualization query language from Tableau Software; Feb. 10, 2010: software that may help in-house counsel make decisions on data; and May 25, 2011: third party report writers in order of number of users.).

An announcement about the changing of the guard at the Association of Corporate Counsel mentioned that during one person’s tenure large law department memberships had increased 89 percent. Wow, impressive!

Wait! We don’t actually know whether that is impressive. The release did not state the time period or the starting number of members from large departments. It left out the definition of a “large law department,” the level of the new members, and their distribution by department. That is to say, if the expansion took place over ten years, it loses its luster. If the ACC started with 50 “large law department members” and added 45 more, that too is a less impressive story than starting with 500 and adding 450 more.

Further, if the ACC deems a “large law department” to have as few as 10 lawyers, the statement has different impact than if the term refers only to departments with more than 100 lawyers.

Courtesy of Aspen, publisher of the two-volume directory of more than 30,000 in-house lawyers at some 8,000 corporations and nonprofit organizations, your law department can get the 2010 directory on CD-ROM. This year’s edition (2011) has more than 2,800 pages and retails for $1,495. For questions about the directory, write Kate Brady.

The CD-ROM will be sent to you at no cost if your law department completes the General Counsel Metrics benchmark survey for the first time during the next 10 days.

To benefit from this special offer, good through September 17th, click on this Aspen GCM link and submit your 6 elements of 2010 staffing and spending data. You will receive both Release 3.0 of the GCM benchmark report (with 500+ participants) and the CD-ROM by early October. Please state at the end of the survey where it asks how you learned about the opportunity that you would like the Aspen Directory of Corporate Counsel or send me an email.

To get Release 3.0 in late September and its benchmark metrics for your industry based on 500+ companies, all at no cost, click here to take the General Counsel Metrics global benchmark survey.

As of this morning, 383 law departments have submitted their six pieces of staffing and spending data. They reported 9,394 attorneys, along with 2,535 paralegals and 5,281 other legal staff (medians of 8, 2, and 3, respectively). At a combined inside and outside spend of $9.6 billion, they supported $2.3 trillion of corporate revenue.

In addition to the 21 basic industries reported on by General Counsel Metrics, Release 3.0 in late September will have at least five specialized industry-segments, such as restaurants, national laboratories, medical devices, universities and others. Once there are six or more participants in an industry segment, the Release creates benchmarks just for them. Also of note, exactly 100 of the respondents are Fortune 500 or larger (more than $4.7 billion in revenue). Of them, 29 weighed in at $20 billion or more; the median revenue of the entire group, however, is $1.4 billion. At the smaller size, 60 of the departments reported corporate revenue of $100 million or less.

Academics conduct research that makes me envious. Consider a detailed study of contracts described in the Acad. Mgt. Rev., Feb. 2011 at 182. Three professors studied 385 contracts Compustar had entered into with buyers of its IT services. (Of interest to me was the statement that lawyers negotiated none of the contracts; “lawyers conducted a final-stage review of the contracts” (at 200).) The article focuses on extendability and early termination provisions in the contracts. To research their effect, the authors coded a number of variables regarding the contractual arrangement and the contracting party. They then conducted regressions to find answers to their hypotheses.

My point here has only a little to do with what this particular empirical research can tell law department managers about contracting activities. My main point is that the methodology of data collection and analysis can uncover what is going on in the world far more insightfully and convincingly than the anecdotal sense of even the most experienced lawyer. Once you count it, analysis it, and run regressions coupled with causal relationships, you reach a much deeper understanding of whatever you study. Academics can do this and they publish their findings. This article illustrates the power of that approach.

If there were an accepted rate to convert the number of a law departments paralegals to lawyer equivalents, such as three paralegals will be deemed to equal one lawyer, and similarly to convert support staff to lawyer equivalents, what would that conversion tell us as a benchmark? First, though, although the math is easy, does the result make any sense? It may be useful to study law departments that have different mixes of lawyers, paralegals and support staff in a comparable, single expression.

I tried this on a group of 22 medical devices companies that have taken part this year in my General Counsel Metrics global benchmark survey. Three paralegals equal a lawyer, for the purposes of this exercise, and five support staff equal a lawyer. I based this rate loosely on compensation as well as contribution and mean no disrespect to non-lawyers, but if the idea of equivalency has legs, there needs to be some translation into a common denominator. I have bruited this idea before (See my post of July 31, 2006: secretaries counted in terms of lawyer-equivalents; June 28, 2006: a proposed standard; and Feb. 4, 2010: lawyer-equivalents.). Try as I might, I could not tease out any pattern or conclusion from the exercise. The various rankings and calculations I tested did not suggest a pattern.

The law departments of large and acclaimed companies sit square in the targets of most benchmark surveys. Perhaps we should call them the Unfortunate 500, deluged as they are with requests for benchmark participation.

All survey sponsors want name brand respondents. Bigger is better; better known is better. Particularly when consultants sponsor the survey, such as the former Hildebrand Law Department Benchmark Survey, the consultants want to sell their services, and larger law departments are thought to be more likely purchasers of services or at least purchasers of more costly services. For these reasons, benchmark respondents may bulge at the large end.

Exacerbating that tilt, small departments may pass over benchmark surveys more than larger departments do because the one, two or three lawyer general counsel feel they have less maneuverability in response to benchmark metrics. On the other side, you can’t hide in a smallish company where cost and headcount pressures are front and center; there may be more urgency to find comparative metrics.