Articles Posted in Benchmarks

Over the three-year span of the survey, the average size of FTSE 500 legal departments surveyed has grown from an average of 25 lawyers to 62 …” Pause, and reflect on the quoted portion about rapid growth (See my post of Dec. 5, 2006 about the end of the quote, about the proportion of work sent out.).

This quote, from Derek Bedlow in Legal Week, Nov. 23, 2006 at 22, refers to the latest Legal Week Intelligence Client Satisfaction Survey, “which questioned 297 general counsel and other senior decision-makers at FTSE 1000 and Alternative Investment Market companies about their law firm hiring policies.” Also, I presume, about law department size.

I doubt that in three years the law departments of the largest UK companies have on average more than doubled (average 25 to average 62 lawyers). The tricky phrase “legal departments surveyed” may betray a statistical artifact if it means “legal departments who responded to the survey.” If a number of large departments waded in this year (or small ones didn’t), the pool may slosh out with such a huge rise, but it is really inconceivable that same-department growth over just three years has been so robust.

The largest law department in New Jersey is the Attorney General’s Division of Law – all 580 lawyers. The Division represents all 16 departments and more than 400 agencies in the New Jersey state government. The NJ Law J., Nov. 27, 2006, explains the difficulties the Division has had keeping up to strength and its work load. “At any given time, the division has more than 25,000 cases pending in federal and state trial and appellate courts and administrative forums. Of these, about 13, 000 are litigation matters, 11,000 are administrative cases and 1,700 are appeals.”

At 580 lawyers and about 9 million residents, the state’s law department has approximately one lawyer for every 15,000 residents. If that ratio applies to the other 49 states and their 292,000,000 residents, then the aggregate in-house legal functions of state government would employ close to 19,000 lawyers.

Selection bias distorts a statistical analysis because of how the data was collected (See my post of Dec. 1, 2006 for an introduction.). Let’s consider some examples in the context of law departments.

Self-selection bias. Law department lawyers with strong opinions, deep interests, or substantial knowledge may be more willing to spend time answering a survey than those lawyers who don’t have any of those characteristics. If a survey asks about pro bono, for example, those who couldn’t care less will probably not bother to respond (See my post of May 14, 2005 suspecting this on a knowledge management survey; and Aug. 26, 2006 generally.).

Selected end-point bias. For example, to maximise a claimed trend of lower outside counsel costs, a law department could start measuring the drop from an unusually high year. Or a compensation survey draws its data from law departments that are generally much larger or much smaller than the surveying department is suspect.

The total revenue of the Fortune 500 in 2005 was $9.1 trillion. Based on the Hildebrandt 2006 Law Department Survey, median lawyers per billion dollars of worldwide revenue the same year was 3.1 (per billion dollars of US revenue it was 4.4 lawyers). Multiplying the Fortune 500 revenue by the worldwide ratio indicates about 28,000 lawyers were employed in-house by the Fortune 500.

We might take one more step and assume that 90 percent of the lawyers who work for Fortune 500 companies are admitted somewhere in the United States (See my post of March 19, 2006 to back that estimate.). That number, about 25,000, should be compared to the 72,000 corporate lawyers identified by ACC in 23,540 companies circa 2004 (See my post of Sept. 25, 2005 on ACC data for 2004.).

It makes sense that more than a third of the in-house corporate counsel in the United States are in the huge companies, the top two percent of the ACC group.

An article by Austan Goolsbee, NY Times, Sept.14, 2006 at C3 describes a researcher’s pitfall: selection bias. If most of someone’s data comes from what surfaces on its own, the data may not be representative of the entire universe of data.

Selection bias applies to what we think we know about management of law departments. The red flag of selection bias alerts us to be careful: we should not to assume that the law department practices that come to light are necessarily representative of those practices in all law departments.

Some general counsel are drawn to publicity (See my post of June 30, 2006 on celebrity law departments.). Some champion a particular cause, be it ADR, pro bono, technology, or whatever. Some companies make news such as those in the Fortune 500 or those where bad things have taken place. All of these selection biases may (probably) distort the true picture of law department practices.

Among the ten lawyers at Palm Inc., one is “on a contract basis because she is working in Germany.” This fact gets no elaboration in the interview of Mary Doyle, general counsel of Palm, published in Law.com In-House Counsel, but the contract basis may have to do with the difficult employment laws of Germany. Or, possibly, the lawyer is working on a trial basis.

Aside from those speculations, note that how a law department classifies contract lawyers can make a major difference in its benchmark numbers. Many of them state results per lawyer (See my post of May 14, 2006 on how problematic this denominator can be.). If a contract lawyer functions interchangeably the same as a full-time employee-at-will lawyer, they should be counted the same. If not, the fully-loaded cost per lawyer will swell, but not for any logical reason. Lawyers per billion of revenue will look better than it should while inside and outside costs per lawyer will look worse.

Truly temporary lawyers, hired from an agency for an expected limited time on a certain project, should be treated differently than a lawyer who is on a contract for reasons having to do with local laws (See my post of Aug. 2, 2006 on Sears and its use of temporary lawyers.).

Many companies operate in more than one market segment. Nevertheless, benchmark surveys treat them as if they operate in only one, in only one industry. How can benchmarkers accurately categorize General Electric, MetLife or Johnson Controls, to name only three companies that have varied operations?

One methodology asks the companies in a benchmark survey to identify their various revenue components. Then, weight the data according to how large a particular component’s revenue bulks in the overall revenue of the company. A company that is 40 percent manufacturing would have its metrics count for more in a manufacturing industry cut than another company where manufacturing accounts for only 20 percent (See my posts on weighting factors of Nov. 24, 2005 regarding the determinants of bonuses; March 25, 2005 and May 31, 2005 regarding client satisfaction scores; Aug. 30, 20065 regarding grid analyses for decisions; Nov. 15, 2005 and Aug. 20, 2006 regarding outside counsel performance; and Nov. 30, 2005 regarding weighted averages generally.).

Gill Hague, a consultant and director of Legal Project Services Ltd., sent some numbers from a Law Society report on lawyers with a Practicing Certificate in central government departments (See my post of April 13, 2006 for more on Hague’s dissertation.). The Law Society – Britain’s equivalent to the American Bar Association – reported 90 lawyers in Government Departments, 125 in Courts, 210 in Government Funded Services, and 1,697 in Crown Prosecution Service.

Hague points out that “there may be other legal professionals – such as Barristers, or qualified Solicitors whose role does not require them to have a Practicing Certificate – working in the employed sectors.”

He closes with mention of a certification that has no counterpart in the United States: “many local government staff in work areas such as Property and Litigation hold other legal qualifications such as Fellowship of the Institute of Legal Executives.”

Along with my previous posts on statistical analysis (See my post of May 31, 2006,), graphical depictions of data (See my post of June 30, 2006 on analysis and depiction tools.), and net score analyses (See my post of Sept. 25, 2006.), consider some of the other calculations that can investigate law department data.

Extrapolate. Extrapolation means to extend a time series of metrics. For example, if you know the number of complaints filed against your company per quarter for the last three years, you can plausibly extrapolate the volume for the coming year.

Quartile. Spending on outside counsel lends itself to an analysis that uses quartiles. If you rank how much your department paid each of its law firms from the largest amount to the smallest amount, the the figure in the middle is the median. Midway between the median and the smallest amount lies the first quartile figure is; the third quartile figure, midway between the median and the largest amount.

A recent survey of European law departments confirmed what many people have known and this blog has supported: as companies increase in size the ratio of numbers of lawyers per billion dollars of revenue decreases. Data in Law Dept. Quarterly, Sept./Nov. 2006 at 19, confirms that economies of scale and scope are hard at work (See my post of July 5, 2006 and references cited.).

This phenomenon shows up in mergers, where the combined legal group emerges much smaller than the two departments that came together (See my post of Sept.13, 2005 about Honeywell and Oracle and the layoffs that result from mergers.).