Claude Shannon, the father of information theory, calculated “that the information content of any event was proportional to the logarithm of its inverse probability of occurrence.” On the off chance that quote leaves you with any doubt, the Economist, April 24, 2010 at 82, says that it means “an unexpected, infrequent event contains much more information than a more regular happening.” If a class action has a one in one thousand chance of being certified, the logarithm of that chance’s inverse (1,000 is the inverse of 1/1,000) is 3. A one in five hundred case has a log of 2.69 so the rarer event has 11 percent more informational value, as that term was defined by Shannon.
We notice deviations from patterns because the oddity contains more “information.” We can learn more from the unusual than from a continued series of normals. Three examples suggest the range of this axiomatic proposition.
Power-law formulas tell us much because they push us to consider unlikely outcomes and enable us to frame their likelihood (See my post of July 25, 2005: bell curve compared to power-law functions; Nov.13, 2005: power laws and the ratios of litigation costs; May. 27, 2007: Zipf’s law; Feb. 24, 2009: explanation of power-law distributions; Feb. 24, 2009: power laws in law department management; and Sept. 27, 2009: my article on the esoterica of power laws.).