Articles Posted in Benchmarks

I looked at data from 272 law departments that had participated so far in this year’s the GCM benchmark survey (with full data) and the 114 that identified their matter management system. Having described the dispersion of those named systems as well as the eight most common (See my post of June 13, 2011: 63 departments in total), I went a step further with the 63 users of the “big eight.” I calculated the total legal spending as a percentage of revenue (Revenue Spend) for each of those 63 legal departments and then by software package.

For all 272 companies, the average was 0.85 percent while the median was 0.40 percent. For all but one of the software packages, average Revenue Spend was less than the average for the entire group. (When I removed those 63 from the group, the averages and medians hardly budged.}

Conclusions will become stronger as hundreds more law departments take part in the survey, but from this preliminary analysis, one could argue that these well-known matter management systems help their users reduce their total legal spend as compared to the average of law departments that have not chosen to license one of them.

A presentation recently by Ralph Schroeder of Hyperion Research Group listed the ten most common key performance indicators (KPIs) and metrics used in intellectual asset management (IAM). Eight of them made complete sense to me, but two left me with questions.

“Number of own patents v. competition” was one of them while the other was “Patent usage (e.g., % patents in products, licensed, etc.)”. Regarding a company’s patent portfolio in comparison to the competition’s portfolio, that seems to be a crude measure rife with assumptions. Crude because sheer numbers of patents owned poorly indicates the quality, strength and centrality to one’s business. Assumption ridden because you have to pick your competitors appropriately and then the specific segments where your patents go head to head. The subjectivity and judgment is obvious.

The patent usage metric also raised questions for me. Your patents that bring in royalties or other benefits through license agreements can be pinned down precisely, but the percentage of your patents in products brings many more challenges. Even more difficult would be to attach revenue generated by particular patents.

Just shy of 300 participating law departments, the GCM benchmark survey has collected responses from 114 of them regarding their matter management system. In declining order of mention, the providers of such software were Mitratech (19), Serengeti (12), LawTrac (8), CT Tymetrix (7), EAG (6), Bridgeway (4), Datacert (4) and LexisNexis (3).

At least 29 other packages were mentioned by one or more law departments, notably SharePoint (3), Pro-Link (3), and Tedesco and Smart Counsel (2 each) (See my post of March 12, 2011: early look at matter management systems based on 142 participants.). Nine law departments reported that they use a custom, internally-developed system.

The first-ever league table for this particular software niche, based on user data rather than vendor statements, will improve steadily as the GCM survey gathers hundreds of more participants. Take the survey (see the icon at upper right) and learn more!

As a metrics wonk, this abuse of numbers bothers me. I published a post today that quoted a consultancy’s claim that you can “save up to 90 percent” of some alternative course of action.

Diet sheds up to 100 pounds! Earn up to $2,000 a day! Pass over up to 95 percent of your competitors!

Data analysts and marketers should not tout extremes. They should give medians and averages. “Up to $100 an hour lower rates” from firms is more realistically stated as “A median reduction of $21 an hour.”

Caution, shoals of math ahead before the port of understanding is reached.

Assume the effective billing rates of outside counsel run at about 50 percent higher than the fully loaded cost per hour of internal counsel — $330 outside to $210 inside would be unremarkable for many U.S. law departments.

Assume the typical law department spends about 50 percent more on outside counsel than on its inside group, which amounts to the common 40/60 split of those two categories of spend.

Come listen and learn while you lunch! I will be holding forth on the latest data from the General Counsel Metrics global benchmark survey. (Have I mentioned that survey here before?) Even better, ask any questions you may have if you call and dial in for the one-hour webinar, arranged by Mitratech, to take place at 1:00 PM Eastern time that Thursday.

Sign up now so that you or someone who works with you can take part in this hands-on discussion of law department benchmarks for spending and staffing. Click on this link for the short webinar registration. While you’re at it, click on the icon upper right, put in your department’s data, and get your report by email in about a month.

A previous hyperpost brought together 10 metaposts on benchmarks (See my post of July 19, 2009: ten metaposts on benchmarks.).

Since then, another 13 have accumulated. Some of these newer ones quite likely repurpose earlier posts that have already been included, many of the posts are recent.

Some metaposts concern individual benchmark metrics (See my post of Oct. 27, 2009: one-to-one ratio of lawyers to support staff with 9 references; April 16, 2010: TLS goes down as revenue increases with 9 references; April 16, 2010: TLS as percentage of revenue with 15 references; Sept. 20, 2010: R&D spend related to legal spend with 8 references; April 3, 2011: all-in cost per hour of internal lawyers with 9 references.).

Performing under his nom-de-blog Rees Morrison, that ever-popular raconteur of ratios, the nation’s nimble narrator of numbers, regales the YouTube world with inimitable insights about indices and magical moments on metrics! This live, uncut performance, filmed al fresco, ad hoc, without makeup or script writers, and away from the klieg lights of the paparazzi, smuggles out a rare glimpse into the wild and wacky world of comprehensive total expenditures as a percentage of corporate revenue, at the median no less!

Mouse here, you millions, crank up the speakers, and settle back for a head-warming three minutes of vintage benchmark boisterousness!

You can relish it early, before Cannes, even well before Sundance, this feel-good, nano-documentary on derecho data. Law department benchmarks will never be the same!

The law department administrator from The Williams Companies, speaking at Mitratech’s Interact 2011 Conference, mentioned that her department of about 70 lawyers averages 1,500 matters a year. That ratio grabbed my attention since I have not seen a metric such as 20-30 matters per lawyer per year. Of course, that metric depends how a law department defines matters (See my post March 26, 2008: definition of “matters.”). Still, assuming Williams only creates matters for reasonably significant activities – reminds me of named hurricanes – at least that is the basis for a comparative benchmark. Even internally it could be revealing.

She went further. Of those 1,500 matters, about 500 have outside counsel. The ratio of one-third of all matters having some fees billed by external counsel offers another metric that is uncommon – but potentially very useful.

At the least, law departments can pay attention to both metrics and give some thought to their own patterns over the years. At some point, and perhaps even now, a benchmark study will suggest what ratios are most common.

When researchers gather data for statistical analysis, there is always some slippage, by which I mean that the results would likely vary within a calculable range if the researchers duplicated the study. Statisticians refer to that variability of results as margin of error or sampling error. Here is a good description of that flux from a recent survey. “The margin of sampling error at the 95% level of confidence is ±4.6 for total lawyers (N=455) and ±9.8 for total students (N=100). This means that if we were to replicate the study, we would expect to get the same results (within 4.6 percentage points for lawyers and 9.8 percentage points for students) 95 times out of 100.” Thus, for example, if 30 percent of the lawyers said they were underpaid, only five percent of the time if a similar survey were conducted over and over would the result be lower than 25.4 percent or higher than 34.6 percent.

All statistical survey findings should explain their margin of error so that those who use the findings can determine their degree of reliability. Smaller samples for the most part have larger margins of error (See my post of Dec. 9, 2005: margin of error and sample size; Aug. 29, 2006: margin of error and subgroups; Aug. 30, 2006; sampling error; April 22, 2007: sampling error; Oct. 31, 2007: formula for margin of error and benchmarks; Feb. 7, 2008: margin of error of a group, yl√n = 4xl√16 = x; and Jan. 11, 2011: synthetic indices and margins of error.).