Headline: Survey says scientists mistrust a large amount of published research/New Scientist by New Scientist Staff and Press Association.  August 21, 2019

Excerpt: “Out of those surveyed, 25 per cent said exaggerated findings, a lack of detail, and poor conclusions make research outputs untrustworthy. ‘There’s always someone trying to pull the wool over your eyes. Within your own field this can be easier to detect, but it’s less easy to determine when scouting subjects that you are less familiar with,’ a materials scientist in the UK told the survey.”

Original Survey Report: Trust In Research Report/ Elsevier June 2019

Earlier this year, Elsevier, an information analytics business, surveyed over 3,000 researchers regarding how trustworthy they found published research (or “output” in the common nomenclature). In response to the question, “Thinking about the various research outputs that you interacted with (or encountered) last week what proportion of the outputs would you consider trustworthy?”, 37% indicated they trusted up to half the outputs; 48% trusted most, and just 14% trusted everything they read. In other words, 86% of researchers rated at least some published research as untrustworthy. The main reasons given were: poor interpretation of the research findings, lack of clarity about the peer review process, and flaws in the methodology (e.g., bad study design).

A Few Rules-of-Thumb on Reading Science Articles:

  • Whenever reading science articles, make sure they provide links to the original published research papers. If not, they are trying to control your interpretation of the findings.

  • Does the article interrogate the research - that is, present views of other scientists on the assumptions, quality and interpretation of the research discussed? If not, consider the article a puff piece and not serious science writing.

  • Does the article appeal to signals of authority, e.g., how much the authors have been published or cited, whether they are “well-respected” or subscribe to the consensus or long-established view? If so, you are dealing with a propaganda piece.

  • If an article is written with conviction or the aim to convert, it is not written in the spirit of science.

As for the spirit of science, Todd Litman says it well:

“Good research is cautious about drawing conclusions, careful to identify uncertainties and avoids exaggerated claims. It demands multiple types of evidence to reach a conclusion. It does not assume that association (things occur together) proves causation (one thing causes another)..” - Todd Litman, Evaluating Research Quality.