The following passages in Joshua Rothman’s “How Does Science Really Work?" inspired this post: 

[Scientists] “are constrained by a central regulation that governs science, which [Michael Strevens] calls the “iron rule of explanation.” The rule is simple: it tells scientists that, “if they are to participate in the scientific enterprise, they must uncover or generate new evidence to argue with”; from there, they must “conduct all disputes with reference to empirical evidence alone.” …Strevens’s rule can feel obvious and underpowered. That’s because it isn’t intellectual but procedural. “The iron rule is focused not on what scientists think,” he writes, “but on what arguments they can make in their official communications.” Still, he maintains, it is “the key to science’s success,” because it “channels hope, anger, envy, ambition, resentment—all the fires fuming in the human heart—to one end: the production of empirical evidence.”  

[Some theorists – subjectivists - point out] that scientists are as hopelessly biased as the rest of us. To this group, the aloofness of science is a smoke screen behind which the inevitable emotions and ideologies hide. Strevens offers a more modest story. The iron rule—“a kind of speech code”—simply created a new way of communicating, and it’s this new way of communicating that created science. The subjectivists are right, he admits, inasmuch as scientists are regular people with a “need to win” and a “determination to come out on top.” But they are wrong to think that subjectivity compromises the scientific enterprise. On the contrary, once subjectivity is channelled by the iron rule, it becomes a vital component of the knowledge machine. It’s this redirected subjectivity—to come out on top, you must follow the iron rule!—that solves science’s “problem of motivation,” giving scientists no choice but “to pursue a single experiment relentlessly, to the last measurable digit, when that digit might be quite meaningless.”

- Joshua Rothman in the October 2, 2020 issue of The New Yorker, quoting Michael Strevens, author of “The Knowledge Machine: How Irrationality Created Modern Science

Strevers’s basic argument is that scientific knowledge grows through application of the “iron rule” of explanation: empirical testing and its documentation in scientific journals. The main reason science is effective is not due to the “unwavering rationality of any individual scientist” but the willingness of scientists as a group to abide by the iron rule. This is how science advances.  

The iron rule requires scientists to base their arguments on empirical evidence. Motivated by love of truth, glory, and tenure, scientists tend to be a critical bunch: they like to pick holes in each others’ arguments and evidence. To defend themselves against this critical onslaught, scientists adhere as much as possible to various procedural norms governing the gathering and analysis of evidence - also known as the scientific method.

Of course, scientists policing themselves is not enough to keep science on its self-correcting trajectory. Research institutions must also ensure that good scientific practices are observed, with balanced reward and punishment systems in place to uphold research quality and integrity.

Next: What, exactly, is the Scientific Method?

References:

 “How Does Science Really Work?” by Joshua Rothman/The New Yorker; September 28, 2020. https://www.newyorker.com/magazine/2020/10/05/how-does-science-really-work  

Institute of Medicine 1992. Responsible Science: Ensuring the Integrity of the Research Process: Volume I. Washington, DC: The National Academies Press. https://doi.org/10.17226/1864.