Many posts on this blog have dipped into the well of the annual Fulbright & Jaworski surveys of litigation data. Each year the firm polls a few more than 400 law departments in the United States and the United Kingdom.
If the responding group year after year remains significantly similar might not the results reflect some degree of self-reference? If one year’s results show a decline in international arbitrations, to imagine one possibility, might the respondents read that and be influenced in their behavior during the following year, or just in how they remember and answer the next year? They hear the echo of their own voices and answer questions to some degree influenced by their own reported behavior.
My question ranges more broadly than F&J’s admirable investigations. We could surmise the same reverberation and influence from the annual ACC/Serengeti surveys. The echo chamber should have little effect on quantities that are easily counted, such as budget figures or staffing numbers. Its risk rises substantially on qualitative assessments, such as regarding the perceived effectiveness of alternative billing arrangements. When respondents have heard their own, collective assessments one year, the bandwagon effect the next year may reinforce beliefs and thus answers in the same direction.
As it happens, this blog has cited the Fulbright survey many times (See my post of March 8, 2005: Fulbright & Jaworski survey; April 8, 2005: lawsuits per billion of revenue; Oct. 19, 2005: measures of law department success; Oct. 27, 2005: median time to resolution of “roughly half a year”; Oct. 27, 2005: litigation costs and settlement rates at trial; Oct. 27, 2005: one quarter of companies did not sue during the year; Oct. 27, 2005: dubious data on litigation costs; Oct. 29, 2005: class action metrics; Oct. 29, 2005: diversity; Oct. 30, 2005: negative qualities of law firms; Oct. 26, 2007: discovery spending; Oct. 29, 2007: litigation matters per inside counsel; Oct. 29, 2007: drop in e-discovery counsel; Oct. 29, 2007: fixed fees and frequency; Oct. 30, 2005 #3: lack of innovation by law firms; Oct. 31, 2007: insurance coverage against nine kinds of litigation; Dec. 6, 2007: discovery costs; Dec. 23, 2008: no data on lawsuits per billion of revenue; Dec. 23, 2008: many sub-$100 million companies have in-house lawyers; Dec. 23, 2008: size and unlisted status; and Dec. 26, 2008: odd data on litigation budgets.).