A previous post defined construct validity (See my post of Dec. 15, 2010: Meyers Briggs claimed to lack construct validity.). Initially, we would need a consensual construct for effective law departments. “A construct is research-based and its meaning is agreed upon by a consensus of professionals qualified in the appropriate field of study.”
Next, to have construct validity, there would need to be a shared agreement on how to quantify “effective” for legal departments. Experienced professionals would have to agree, in the main, that the instrument or data that purports to measure effectiveness actually reflects that real-world phenomenon.
It’s easy to toss off a definition of an effective law department but it is devilishly hard to measure what that means or to compare two departments objectively. We are not near a common definition of effective law departments let alone measurements that prove the situation (See my post of Sept. 10, 2005: an index of law department effectiveness; Oct. 1, 2005: efficient and effective as models; and Feb. 22, 2010: larger departments gain effectiveness.).