Surveys that purport to show comparisons year to year need to disclose overlapping respondents

When a survey announces that the average or median figure for something one year was X and the same figure rose or fell Y percent the next year, someone who wants to make use of that data must have confidence that the survey populations both years were either very large or reasonably similar. Otherwise, any change may be mostly regression to the mean (See my post of Feb. 7, 2006 for that concept.) or reflect a different set of participants in year two.

The smaller the set of participants, the greater the built-in variability. The concern with consistent data sets arises particularly if a survey reports averages, because on large result can put off the comparison.