Surveys that show one-third up, one-third down, and one-third neutral may tell us very little

My fascination with figures doesn’t blind me, I hope, to misuses of metrics. One potential distortion occurs when a survey reports that “one-third of all respondents think hourly billing is doomed” to support their view that massive change is underway. Perhaps the billing rapture is upon us, but if one third also said “hourly billing will outlast landfill plastic” and one third come down firmly and absolutely on the side of having no clue, the heralded finding loses credibility (See my post of Sept. 9, 2009: one-third splits are predictable on many survey questions.).

For example, CPA’s State of the IP Industry Survey www.ipreview.com last year concluded that law firms need to educate their clients about the value of their IP assets because IP budgets were under pressure. “This approach seems sensible given than more than one third (37%) of the in-house professionals surveyed said that they plan to spend less in 2009 on protecting their IP assets than they did in the previous year.”

Did an equal (or perhaps even larger) percentage intend to spend more? Did one out of three or so forecast a steady expenditure despite the downturn?