Articles Posted in Benchmarks

“I observe that when we mention any great number, such as a thousand, the mind has generally no adequate idea of it, but only a power of producing such an idea by its adequate idea of the decimals, under which the number is comprehended.” Stanislas Dehaene, The Number Sense: how the mind creates mathematics (Oxford 2d ed. 2011) at 79, quoted David Hume, A Treatise of Human Nature, and set me to agreeing.

We can’t grasp 10,000 very well (one quarter of the attendees at Citifield?) let alone one million dollars. Evolutionary psychologists would say that homo sapiens on the great plains of Africa did not need to count above a few score, so our brains never evolved to cope with numbers a thousand or more times bigger. Perhaps decimals help some people. Probably, though, most of us toss around large numbers – GDPs in the trillions, revenues in the billions, fees in the millions – but aren’t really comfortable with the magnitude of numbers at those scales.

At 219 claimed participants in the 2011 HBR consulting metrics report, the number has dropped, since press releases in the past make much of increases (See my post of Oct. 20, 2010: HildebrandtBakerRobbins press release stated 10% increase in participants.). (There is no mention of what number of those 219 completed only the compensation portion.)

In fact, last year’s report had 252 companies, so the loss of 31 leaves one wondering about the causes of the 15 percent decline.

Maybe it’s the hefty cost? Law departments without deep pockets can’t afford two or three thousand dollars for a survey that averages only 10 companies per industry. Maybe it’s the labyrinthine survey instrument? Managers may elect not to devote the man-hours to trying to complete, estimate, or flat out guess at the numbers. Maybe it reflects organizational disarray? The tumult and disorder of being flung out of Thomson Reuters probably took its toll. Maybe modest-sized law departments, which means nearly everyone out there, don’t find data from giant large departments compelling, nor are they wowed by 11 digit revenue ($10B and above)?

Scrupulous surveyors know that if one person answers all the questions, the methodological concern for “response bias” lurks. Better data comes from multiple sources. Rather than have only the administrator of a legal department, for example, answer questions about practices in a law department – numeric answers are not as vulnerable to deliberate or unintentional skewing – different parts should be completed by different people. Sometimes referred to as “same source bias,” it makes the results less reliable.

This presents problems more when the questions have to do with qualitative answers, such as “To what degree do your internal clients bring your lawyers in on time?’ and less when questions concern countable, confirmable numbers, such as “How many bankruptcy cases were pending at the end of the year?”

ACC and Empsight have coordinated for several years on surveys of member law departments. As a company, Empsight focuses on compensation data, at least as summarized by their website: “Empsight International, LLC is a human resource consulting firm which helps employers make better decisions about their investment in people. Our primary focus is on conducting compensation surveys in niche markets, domestically and internationally.” Even so, the survey report appends some staffing and spending metrics.

This year’s CLCM Law survey for large law departments, as it is called, had 186 participants, with average revenue of $25.7 billion and a median of $12.8 billion. Those participating law departments reported 7,481 attorneys and 13,720 total headcount.

Last year that survey had 30 fewer participants (156), with average revenue in excess of $23 billion. A year before that it covered 170 companies (See my post of Nov. 25, 2009: median revenue of $12.9 billion dollars, and 7,748 lawyers out of 13,558 headcount.) while two years before that (in 2007) 205 participated (See my post of Nov. 6, 2007: large department survey reported on 205 companies.).

The general counsel of inefficient law departments don’t want to provide comparative data that confirm they are operating inefficiently. They are probably less likely to choose to participate in benchmark surveys than their fitter counterparts. Not that anyone sees any specific company’s numbers, because the data are aggregated and normalized. Even so, if companies with less flattering benchmark figures systematically don’t take part, perhaps claiming ill-advised concerns about confidentiality, their numbers are not in the data and the benchmark results conceal mediocrity. Published benchmarks, therefore, may be better than what actually and typically prevail.

This is the mischief of self-selection bias. Well-managed departments want to prove their quality so they take part disproportionately in benchmark surveys. What does self-selection cause? Some foundational findings, like total legal spending as a proportion of corporate revenue, may be higher than the benchmarkers state. If only the lean ones weigh themselves, the world is fatter than the resulting medians and averages. There is no mechanism yet by which a large group of law departments must provide figures that test this hypothesis of self selection.

The best we can hope for until then is to have trade groups reach out and urge wider participation among their members. Or, to have benchmarking be seen as simply what everyone does. Sheer numbers of participants means the results are likely to dig deeper and be more representative.

If you are thinking about participating in the no-cost, quick and confidential online survey, you might want to see whether you have peers of about your size in your industry. Release 4.0 will go out in early December with more than 750 law departments. Covering 22 industries and five revenue ranges, the table that you can download will answer your question immediately.

Click here to download the industry-participant table. Look for your industry row and then the column that includes your revenue for fiscal year 2010.

To take the quick survey and get your report, click on this survey link.

We take for granted that fundamental terms in benchmark surveys and reports have clear meanings understood and applied by all. Lord, what fools these mortals be!

Setting aside statistical terms, below are 12 that deserve careful definition by surveyors and scrupulous adherence by respondents so that benchmark metrics may be most useful. A number of posts on this blog have tried to clarify several of these terms.

1. Chargeable hours (See my post of May 16, 2006: in-house chargeable hours.).

The 409 companies so far in the General Counsel Metrics (GCM) global survey report that all together they have 9,900 lawyers (median 7) and 8,212 other legal staff (median 6), which includes paralegals. Their combined legal spend totals $10.2 billion (with a median of $5.6 million) and the revenue of the companies they serve comes to $2.4 trillion (a median of $1.5 billion).

Included in the set so far are law departments from 29 countries, with the US and Canada accounting for seven out of ten participants. Representatives from less-numerous countries include Bermuda, Kuwait, Norway and the UAE. Release 3.0 will have at least 22 industries, including special breakouts for medical devices, national labs, banks (which were combined under Financial Services last year), and possibly two or three more if there are at least six companies in the segment. The largest representation falls under Manufacturing, with 62 departments.

Learn benchmarks for your law department’s staffing and spending. Click here to take the eight-minute confidential survey and get your own confidential, 55-page report by e-mail.

When we measure something, such as the time it takes to review advertising brochures, what takes place? As the author David Deutsch writes, “One can claim to have measured a quantity only when one has an explanatory theory of how and why the measurement procedure should reveal its value, and with what accuracy.” (at 317). This blog presents 10 ripples from a decision to count and track something.

We evidence authority, in that someone says, “do it” and others follow. Even if the same person who conceives the measurement does it herself, she shows authority over herself: self-discipline.

We introduce a time period for measurement, even if we don’t decide an end point explicitly at the start (“Let’s keep track until we see a pattern.”).

Do you have blazoned on your office wall “for every dollar you spend, the company’s sales force has to generate $10 of additional revenue to cover that expense”? The authors, writing in ACC Docket, Nov. 2010 at 81, actually mean “profit,” not “revenue” but the point remains dramatic. If plus or minus a bit, one percent of revenue goes to total legal expenses and if plus or minus 10 percent of revenue makes it to the bottom line as profit, then every legal dollar costs $10 of profit.

The authors add a fillip: “Never spend the company’s money in a way that you would not spend your own. In fact, treat the company’s assets more dearly than your own.” Does that apply when the in-house lawyer takes a car service or flies business class or treats outside counsel to a fancy dinner? We all find it quite easy to spend reimbursed “company money” much more liberally than our own hard-earned income.

Anyway, legal spend per unit of revenue reigns as the Magi of Metrics, but in the bottom-line world of shareholder return and working capital, legal spend per unit of profit should be the king (See my post of March 4, 2008: profit margins and lawyers per billion; May 24, 2005: profit margin as a benchmark denominator; April 4, 2006 #1: “earn a dollar to pay a 3-cent bill of outside counsel”; May 23, 2007: profit per in-house lawyer; and Feb. 15, 2010: $4,300 per hour in revenue typically needed to pay for one hour of outside counsel.).