When economists publish articles based on results from analyzed datasets, they publish the dataset online so that others can test it or make use of it in other ways, according to Ian Ayres, Super-Crunchers: Why Thinking-By-Numbers is the New Way to be Smart (Bantam 2007). It would be wonderful if benchmark data about law departments, with confidentiality preserved, could at least be partially made available for everyone to study.
We need a creative commons license by respondents to surveys: I will give you my data but it must be made available for all others who can make use of it (See my post of June 6, 2006: Empirical Legal Studies.). That way there would be a shared pool of raw metrics, available to all researchers, with company names and industries deleted or somehow coded so that no one could figure out which law department a particular number comes from. A way around this worry, perhaps, would be to post online only ratios, not absolute numbers.
My dream is probably a long way off, although I happen to believe that most data of law departments, even if shouted from the rooftops, won’t help another law department (See my post of April 15, 2007: what information should law departments be concerned about disclosing.). The biggest obstacle is that those who collect the data make money from it and view the data as a source of proprietary gain from publicity and knowledge. It should be the analysis and clarity of graphics that distinguish someone, not the raw data itself.
At the least, the law departments of this world would be better off if there were common definitions of the terms used in data collections and surveys. It would be even better if there were a standard format for them to submit data, something like the Common Application for college admission.