“Good research is cautious about drawing conclusions, careful to identify uncertainties and avoids exaggerated claims. It demands multiple types of evidence to reach a conclusion. It does not assume that association (things occur together) proves causation (one thing causes another). Bad research often contains jumps in logic, spurious arguments, and non-sequiturs (‘it does not follow’).” Todd Litman

Good research papers are imbued with the spirit of humility. Their authors seriously consider alternative explanations for their research findings and do not overstate the significance of these findings. Based on these two criteria alone, let’s look at Testing Theories of American Politics: Elites, Interest Groups, and Average Citizens by Martin Gilens and Benjamin I. Page, which created quite a stir in 2014 when it was published.

In this paper, the authors looked at pro/con survey responses regarding support for various federal policy proposals and then determined if the proposals were later adopted as US government policy. Survey respondents were classified as either median-income “average citizens” or “economic elites” (top 10% income bracket, serving as a proxy for the very wealthy). The authors’ data analysis revealed that economic elites and average citizens had the same policy opinions about 78% of the time. When both groups supported a policy change, the change was later adopted a bit less than half the time.  When they disagreed, policy changes supported by elites were occasionally adopted but policies supported by average citizens alone were almost never adopted. Thus, according to Gilens and Page, when “a majority of citizens disagrees with economic elites…they generally lose”* (p. 576). They conclude that “majorities of the American public actually have little influence over the policies our government adopts” and “a small number of affluent Americans” have too much influence. (pp. 576-577)

Next up: did the authors seriously address alternative interpretations of their data and are their conclusions warranted by their findings?

* The Gilens and Page paper also looks at the influence of business and mass public interest groups on government policies, which I will not be addressing in these posts.

References:

Martin Gilens and Benjamin I. Page Testing Theories of American Politics: Elites, Interest Groups, and Average Citizens  Perspectives on Politics September 2014 | Vol. 12/No. 3, 564-581 doi:10.1017/S1537592714001595

Todd Litman Evaluating Research Quality:  Guidelines for Scholarship  22 February 2012 Victoria Transport Policy Institute