Good research is cautious about drawing conclusions, careful to identify uncertainties. It demands multiple types of evidence to reach a conclusion. It does not assume that association (things occur together) proves causation (one thing causes another)… Bad research often uses accurate data but manipulates the information to support a particular conclusion. Questions can be defined, statistics selected and analysis structured to reach a desired outcome. Alternative perspectives and data can be ignored or distorted. -- Todd Litman Evaluating Research Quality: Guidelines for Scholarship October 4, 2019; Victoria Transport Policy Institute
Keeping the above observations in mind, let’s look at Testing Theories of American Politics: Elites, Interest Groups, and Average Citizens by Martin Gilens and Benjamin I. Page, which created quite a stir in 2014 when it was published. In this paper, Gilens and Page analyzed survey data on public support for various policy proposals, some of which were later adopted by the US government.. Survey respondents were classified as either median-income “average citizens” or “economic elites” (top 10% income bracket, serving as a proxy for the very wealthy). The authors’ data analysis revealed that economic elites and average citizens had the same policy opinions about 78% of the time. When both groups supported a policy change, the change was later adopted a bit less than half the time. When they disagreed (and holding other factors constant), 18% of policy changes supported by elites alone were adopted, but policies changes supported only by average citizens were almost never adopted. Thus, according to Gilens and Page, when “a majority of citizens disagrees with economic elites…they generally lose” (p. 576).
Gilens and Page conclude that average citizens lack influence on government policy. They argue that if average citizens get policies they support, it’s only because economic elites also favor the policies. Although they decline to speculate how elites actually exercise influence, Gilens and Page approvingly quote the Communist Manifesto on how the modern capitalist state exists to serve the material interests of the dominant classes. They further argue that their findings support theoretical models of “Economic-Elite Domination” and “Biased Pluralism”, in which the economic self-interest is central to elite policy support. Bottom line: policy support reflects self-interest and average folks lack influence because the state serves the rich.
Unfortunately, Gilens and Page do not seriously consider potential confounders in their analysis of the data. They identify respondent income as the variable of interest, but pooh-pooh the possibility that income may be a proxy for other factors, such as age or education, which are known to influence political opinion. For example, in “Fault Lines in Our Democracy: Civic Knowledge, Voting Behavior, and Civic Engagement in the United States”, Richard Coley and Andrew Sum note that “the oldest, most highly educated, and highest income” individuals were by far the most knowledgeable about policy issues. Coley and Sum also found that just 7% of college graduates expressed no interest in political affairs, compared to 30% of high school graduates. And, as William Galston documents in "Civic Education and Political Participation", civic knowledge can alter opinions on specific issues, such as immigration policy.
Affluent Americans are more likely to be college graduates than people at lower income levels (who tend to be younger and either still in school or just starting out on a career path). Since federal policy makers also tend to be college graduates, could it not be possible that the greater political alignment of affluent citizens and policy makers reflects similarities other than income? Similarities such as greater civic engagement and knowledge than the “ordinary citizen”? Gilens and Page refer obliquely to this possibility but only to ridicule it:
“Perhaps economic elites and interest group leaders enjoy greater policy expertise than the average citizen does. Perhaps they know better which policies will benefit everyone, and perhaps they seek the common good, rather than selfish ends, when deciding which policies to support.
But we tend to doubt it. We believe instead that— collectively—ordinary citizens generally know their own values and interests pretty well, and that their expressed policy preferences are worthy of respect.”( p.156)
Gilens and Page also treat average citizens and economic elites as though they were two distinct groups. But they’re not. According to multiyear tax return data, over half of American householders reach the top 10% income bracket for one or more years by age 60 (over two-thirds reach the top 20% of the income distribution). If getting into the top 10% counts as being an economic elite, then over half of ordinary citizens become economic elites at some point in their lives (and over two-thirds get to be near-elites). Sorta muddies the water.
Additional shortcomings of the Gilens and Page paper include their reliance on old data and tendency to overstate the implications of their findings. To wit:
“The paper is a provocative one, and there’s sure to be a lot of debate among political scientists about whether it wholly supports the authors’ claims. One issue is that their survey data is pretty old: it covers the period from 1982 to 2002. (On the other hand, it hardly seems likely that the influence of the affluent has declined in the past decade.) Another issue is that, in a statistical sense, the explanatory power of some of the equations that Gilens and Page use is weak. For example, the three-variable probability model that I referred to above explains less than ten per cent of the variation in the data. (For you statistical wonks, R-squared = 0.074.)’ - “Is America an Oligarchy?” John Cassidy/The New Yorker April 18, 2014
Despite all its flaws, the Gilens and Page paper continues to rack up citations (1897 so far, per Google Scholar) and has achieved the status of established science within some circles. For instance, this from a 2019 New York Times opinion piece by Malka Older: “In 2014, a Princeton study by Martin Gilens and Benjamin I. Page found that the United States is an oligarchy, not a democracy, with policy driven by the economic elite and business interests.” Gilens and Page found no such thing.
—
Highly Recommended: Todd Litman Evaluating Research Quality: Guidelines for Scholarship October 4, 2019; Victoria Transport Policy Institute
Reference: Gilens, M. and Page, B.I. Testing Theories of American Politics: Elites, Interest Groups, and Average Citizens Perspectives on Politics September 2014 | Vol. 12/No. 3, 564-58. DOI: https://doi.org/10.1017/S1537592714001595