Articles Posted in Benchmarks

My post of Dec. 7, 2005 gives the background of a 2003 study performed for the American Arbitration Association (AAA), summarized in the ACC Docket, July/Aug. 2004 at 93. The summary ventured down an interesting tangent, stating that “the price/earnings ratios (often thought of, among other things, as a measure or indicator of stockholder confidence in the management of a company) for the ‘most dispute-wise’ companies averaged 28 percent higher than the mean for all the publicly-held companies in this survey, and 68 percent higher than the mean for companies in the ‘least dispute-wise’ category” (id. at 100).

I find it hard to believe that litigation management approaches so significantly influence market judgments, since litigation spending for large companies rarely exceeds a half percent of revenue. My hunch is that some other, shared factor – perhaps overall quality of management – accounts for this finding.

More fundamentally, would research find a negative correlation between P/E ratios and total legal spending? That is, companies that manage legal costs down manage profitability up.

Two years ago, you surveyed your 50 key law firms for their satisfaction with you (bet you hadn’t thought of doing that!), and the median rating was 4.7. This year, with the same 50 firms, that rating rose to 4.8, an increase of 2.1 percent. Can you pat yourself on the back, or does the improvement of 0.1 mean nothing statistically?

The margin of error in a survey, which is the amount by which a particular score could be plus or minus the result found because of non-methodological variation, depends on the number of people who respond. (Note: it is critical that respondents be chosen randomly – and take part without any unrepresentative patterns – so that the survey results can be generalized to the whole population.)

How well the sample represents the population is gauged by two important statistics – the survey’s margin of error and confidence level. They tell us how well the sample represents the entire population. For example, a survey may have a margin of error of plus or minus 3 percent at a 95 percent level of confidence. This means that if the survey were conducted 100 times, the data would be within three percentage points above or below the percentage reported in 95 of the 100 surveys.

Software designed to make a mass of information more meaningful by displaying it graphically holds promise for law departments. Advanced departments, awash in various kinds of data, will find and display data patterns more effectively with visualization software. Let’s consider some possibilities, such as if a law department:

 Has its lawyers track their time. Portrayals of those time records visually would more quickly show imbalances of workloads by level of lawyer or for particular business units.

 Manages hundreds of lawsuits. A map color-coded according to the number of cases pending in a state could have columns, the height of which shows the reserves set for cases in that state.

Three research firms – the Corporate Library, Institutional Shareholder Services (ISS), and Governance Metrics International – rate companies for their quality of governance (Fortune, Nov.14, 2005 at 46). Each rating firm uses its own methodology and data sets to grade companies on the important set of activities loosely termed “governance.” People pay for these ratings and insights because the risks from poor governance affect insurers, investors, regulators, lenders and others.

The risks from poor legal advice also loom for companies, and also affect the same constituents, yet to my knowledge no third party assesses companies comparatively on the quality of their legal staff, the adequacy of their in-house legal resources, the legal surefootedness of their outside counsel, the protections provided their intellectual property, the liabilities faced by them through regulatory and judicial proceedings, and the adroitness of management practices. (See my post of Sept. 10, 2005 on Chief Governance Officers and law departments and my meta-post on risk management of Nov. 15, 2005.)

If analysts can make money comparing something as amorphous as governance, why don’t they evaluate legal preparedness?

Benchmark metrics typically come in three forms: averages, medians and weighted averages. Think of 11 law departments each pooling data on their percentage of certified paralegals. For the average, total the 11 percentages and divide by 11.

For the median, rank the 11 figures from highest to lowest and pick the middle one (or average the two middle ones if there is an even number of figures).

For the weighted average, add up all the paralegals of the 11 departments and all the certified paralegals and divide the certified total by the paralegal total.

Like a 49’er looking for gold, I pan through streams of on-line material hoping to find nuggets that glitter for law department managers. The quantitative ore I often find pertains to patents. Why this online nexus?

Patent filings are public information, as are patent law suits. Another reason is that active trade groups undertake research, like the American Intellectual Property Lawyers Association (AIPLA) and the International Trademark Association (INTA). Third, the massive spending in IP creation and maintenance, as well as the stupendous costs sometimes to businesses if they infringe, both in settlements or judgments and in freedom to operate, gives research in the field a hefty ROI. Also, many law school professors specialize in the IP field, unlike such in-house staples as the Foreign Corrupt Practices Act, litigation management, or contracts negotiation.

The term “fractal” describes nature where small parts resemble the whole, such as a bay’s indentations resembling those of a long shore line (Fortune, July 11, 2005 at 99-100). The pattern of invoices arriving at a law department in a month looks like the quarterly pattern and the latter looks like the annual pattern. The hours clocked in a month by a law department’s lawyers look similar in distribution to those logged during a quarter or a year.

The smaller parts often relate to the larger groups according to what’s called a “power law.” (See my post of July 25, 2005 on power laws.) A power-law distribution implies, for example, that the likelihood of a lawsuit that costs $1 million can be predicted from the frequency of lawsuits costing $500,000, and that the same ratio applies to $250,000 cases. Often, each level higher is one fourth less common.

Bell curve thinking (See my post of Oct. 24, 2005 on its usefulness to law department managers.) demotes the impact of major events, because they are far out on the tail and deemed so unlikely. Power-law thinking reminds us that big hits – while certainly less likely than run-of-the-mill legal expenses – remain a real and calculable possibility.

Thus proclaims BTI Consulting Group (Law Practice, Vol. 31, Oct./Nov. 2005 at 11). According to that research firm, a perfect storm of pricing pressures accounts for this: regulatory changes, heightened risks, increased litigiousness and “marked decreases in in-house legal staff.”

The Hildebrandt 2005 U.S. Law Department Survey, which includes mostly large companies, reports that outside counsel spending increased by 5 percent between its 2004 and 2005 surveys. Altman Weil’s 2005 survey of law departments reported that outside expenditures actually fell one percent from its corresponding 2004 figure (Legal Week, Vol. 7, Oct. 13, 2005 at 84).

These are dramatic differences in reported spending, and I suspect BTI has overstated the amount of the increase.

Most law departments have available the data they need to create benchmarks going back three to five years. As an example, the law department can count its total headcount on December 31st of each year and use the revenue for that year to calculate legal staff per billion of revenue. All of the common benchmarks permit retroactive calculation.

If a law department does not want to spend money for a survey or conduct a bespoke benchmark project, it can still create its own set of historical data, what I call time series benchmarks, and see whether trends point in the right direction. {See my post on July 25, 2005 about embedding numeric goals and April 9, 2005 comparing law to other staff functions, and Aug. 3, 2005 on patent benchmarks.)

ACC and its co-sponsor, Lexis Nexis, reported survey results in 2004 from 1,814 law department responses.

The summary states: “The average per month budget for legal research is nearly $19,000, with the median at $5,000. The budget for online and dial-up services averages around $1,000 annually, making it only a small portion of the total annual legal research budget.” (Confused?)

As described in an earlier post, the median inside budget of the respondents was about $840,000 a year – which the post challenges as being too low, meaning “legal research” at $60,000 per year (12 times the $5,000 figure) would account for 7 percent of the inside budget. That has got to be too high a figure.