Articles Posted in Benchmarks

The Institute for the Advancement of the American Legal System (IAALS), part of the University of Denver, recently published a survey report. www.du.edu/legalinstitute/form-chieflegal-success.html The academics who conducted the survey of legal departments excel in their background explanation of how they accumulated and organized the data in the report. All surveyors should emulate such methodological clarity and completeness.

Source of Population Sent the Survey: The IAALS obtained the mailing list of the Association of Corporate Counsel (ACC) and removed duplicates or multiple names for the same company. Once the list was ready for survey administration, it contained 5,257 individual email addresses. All surveys should clearly describe the group to whom they sent surveys.

Effort. The IAALS, teaming with survey specialists, sent the group four e-mails during November 2009 through January 2010. All four e-mails contained instructions for requesting a hard-copy version of the survey. All surveyors should explain what they did to obtain their data.

According to the headquarters country they stated, I assigned each of the 455 law departments in the first release of the General Counsel Metrics benchmark report to one of seven regions. The pattern that became apparent shows lopsided participation.

North America – 281; Continental Europe – 87; Central Europe and Russia (CER) – 30; UK/Ireland – 27; AsiaPacific – 18; Latin and South America – 14; and the Middle East and Indian subcontinent – 5. Not a single participant from Africa.

The imbalance owes mostly to my consulting career and contacts – US for largest part – and the geographic distribution of organizations that have helped me invite law departments. At a deeper level, however, the English speaking countries have more general counsel who are familiar with benchmark studies, assured of their confidentiality, and appreciative of their informational value. In time, that acceptance will spread.

My most recent column for InsideCounsel, under the nom de plume of “Morrison on Metrics,” gave data on the number of participants in the General Counsel Metrics global benchmark survey who requested anonymity. I speculated on several reasons why they did not want to have the name of their company disclosed.

A few days ago, instead of speculating I asked a new “stealth” participant. The answer was simple: the CEO must review all references to the company that will be made public, such as listing the company as a participant in an benchmark study, and the general counsel did not want to use up a chit to seek permission. Ergo, sub silentio.

This is but one of the 60-page report’s many graphical pages, showing quartiles and other metrics. The next release will cover well more than 500 legal departments, so click here and get your report in its entirety in July.

Clicking on the graphic below will not get you to the survey!

Ranting about methodological shortcomings of surveys doesn’t help me much. If I pick at this survey because it uses misleading averages, that survey because it hides the number of participants, and another survey because its sample size is very small, other collectors of survey data may not understand in the overall what to do right. So, let me state positively what I believe a report based on data from law departments should describe about its participants (See my post of July 21, 2008: survey methodology with 40 references, 25 internal references.).

1. How many people in law departments did the surveyor invite?
2. What was the source of the invitation group, such as all law departments of the Fortune 500, members of ACC, general counsel who belong to the New Jersey General Counsel group?

Respondents to a survey a few months back indicated the three types of cases most commonly litigated by the company. The most frequently selected were “contract disputes” (54%), “employment discrimination” (41%), and “personal injury” (26%). In addition, “complex commercial,” “product liability,” and “intellectual property” were reported by 19%, 18%, and 17% of respondents, respectively. No other case type was indicated by more than 13% of respondents.

This data comes from responses of 367 general counsel to a survey conducted into early 2010 by the Institute for the Advancement of the American Legal System (IAALS). The report is available online. I suppose “personal injury” is primarily slip and fall cases.

The data gives some idea of case types that lend themselves to bundles, such as for fixed fee proposals.

Many general counsel appreciate benchmark efforts. More than 300 have done so in the recently released General Counsel Metrics, LLC Global Benchmark Survey.

But, what about total participation rates? Of the three other benchmark surveys that are available for purchase, out of the total of 515 participants listed in their most recent reports, I counted 53 that completed two of the surveys. More surprisingly, four law departments completed all three of them. It is possible that some of the participants also took part in one or more private surveys that are not available to the public, through a trade association or consulting firm like mine, and do not release the names of the participating companies.

Just in case you were wondering, of the group of companies in the other benchmark reports, more than 75 have taken part so far in this year’s General Counsel Metrics, LLC survey. Hundreds of law departments new to benchmarking appear to have tried it out.

The chief patent counsel at Unilever, Matthew Goodwin, directs “one hundred people in seven offices worldwide,” according to InsideCounsel,June 2010 at 44. That is all the piece says about metrics but with a few minutes research what can we build off that single quote?

Unilever’s revenue in 2009 was about $57 billion, so it had roughly two patent staff for every billion dollars of revenue. Unilever had about 163,000 employees, so one patent staff for every 1,630 employees. The company’s patent portfolio (granted and in various stages of prosecution) is on the order of 25,000 patents, which means approximately one patent staff for every 250 records in the portfolio. If we knew a typical ratio of lawyers to non-lawyer patent staff, these rough-hewn benchmarks would double since we could calculate each of them per attorney. The seven locations invite more metrics.

In short, we can generate an array of metrics from a single “datalet” and a bit of online research. Now, to build on that start!

In the first group of 445 participants in the General Counsel Metrics, LLC global benchmark report, 43 of them asked to remain anonymous (less than 10%). They did not want the first release of the report to list them as a respondent. For several reasons, I surmise, they requested anonymity.

Some were not the general counsel, and they might have been concerned about their right to disclose proprietary information. One or two were in the finance function who might not want their brethren in legal to know they were checking on its performance. Some were general counsel who might be unsure whether their metrics will put them in a good light. If your company remains unnamed, no CFO or CEO can get wind of it and put you on the spot for how your spending compares to the spending of other companies in the industry. Some also may have been unsure about the bona fides of an innovative benchmarking initiative and therefore cloaked themselves in secrecy.

Still curious, I dug into the nameless participants a bit more. Eighteen of the 43, at least, are non-US companies. So, aside from levels of authority, considerations about proprietary leaks, and internal politics, levels of cultural comfort caused some of the participants to want to fly under the radar.

In the first group of 445 participants in the General Counsel Metrics, LLC global benchmark report, 43 of them asked to remain anonymous (less than 10%). They did not want the first release of the report to list them as a respondent. For several reasons, I surmise, they requested anonymity.

Some were not the general counsel, and they might have been concerned about their right to disclose proprietary information. One or two were in the finance function who might not want their brethren in legal to know they were checking on its performance. Some were general counsel who might be unsure whether their metrics will put them in a good light. If your company remains unnamed, no CFO or CEO can get wind of it and put you on the spot for how your spending compares to the spending of other companies in the industry. Some also may have been unsure about the bona fides of an innovative benchmarking initiative and therefore cloaked themselves in secrecy.

Still curious, I dug into the nameless participants a bit more. Eighteen of the 43, at least, are non-US companies. So, aside from levels of authority, considerations about proprietary leaks, and internal politics, levels of cultural comfort caused some of the participants to want to fly under the radar.