Human Science
Inquiry
Peer review
A principle of positivism is that research should be public. The idea is that many eyes will catch all manner of incorrect research. We can see the research performed. We can repeat (“replicate”) a study for ourselves and confirm the results.[1]
In practice, however, especially with much research accessible only through very expensive subscriptions that academic libraries can ill afford, we rely on methodology sections and peer review. Methodology sections detail how we conducted our research. Peer review relies on other scholars to review our work:
Researchers commonly refer to peer review as the “gold standard,” which makes it seem as if a peer-reviewed paper — one sent by journal editors to experts in the field who assess and critique it before publication — must be legitimate, and one that’s not reviewed must be untrustworthy. But peer review, a practice dating to the 17th century, is neither golden nor standardized. Studies have shown that journal editors prefer reviewers of the same gender, that women are underrepresented in the peer review process, and that reviewers tend to be influenced by demographic factors like the author’s gender or institutional affiliation. Shoddy work often makes it past peer reviewers, while excellent research has been shot down. Peer reviewers often fail to detect bad research, conflicts of interest and corporate ghostwriting.
Meanwhile, bad actors exploit the process for professional or financial gain, leveraging peer review to mislead decision-makers. . . .
Peer review has also sometimes stymied important research. Senior scientists are more likely to be asked to assess submissions, and they can shoot down articles that conflict with their own views. As a result, peer review can act as a shield to protect the status quo and suppress research viewed as radical or contrary to the established perspectives of referees.[2]
Paul Thacker and Jon Tennant do not argue against peer review. They merely point out many problems.[3] The trouble is we don’t have a solution for these or other problems.
Another problem that Thacker and Tennant fail to mention is that the experts chosen to review work are usually from the same disciplines as the author who submitted it, when often an interdisciplinary or transdisciplinary approach is needed.
Particularly in the social sciences, where boundaries between disciplines are ill-defined and often transgressed, authors can make bonehead mistakes. Reviewers from their same disciplines often won’t know any better to catch them.
For example, I have seen a study detailing what I recognized from my Introduction to Communication class way back when as linear communication. It was about the first thing they taught us. The second thing they taught us is that nobody goes by that anymore. Try to imagine how wide my eyes were as I read this uncited description.
In another example, in the social sciences, in the question of ‘nature versus nurture,’ we overwhelmingly presume nurture, that people are largely products of their environments rather than their biologies. There are exceptions, but social scientists by their very nature will not focus on biology; it’s not what they’re there to study.
Imagine, then, my surprise when I took a human development class where the textbook emphasized nature over nurture. At the very least, the authors should have explained why they were emphasizing nature and should have addressed arguments for nurture. They didn’t.
These errors were not caught. There are, most assuredly, many others.
The original idea that research should be public to be properly vetted[4] has become impractical as knowledge has advanced and become increasingly specialized, well beyond layman levels. That doesn’t mean it should be private, as with expensive subscriptions that make access difficult even for qualified scholars. And while peer review is subject to the same foibles as the research it reviews, that doesn’t mean it can be dispensed with.[5] As with many things, we need to do better.
Paul D. Thacker and Jon Tennant, “Why we shouldn’t take peer review as the ‘gold standard,’” Washington Post, August 1, 2019, https://www.washingtonpost.com/outlook/why-we-shouldnt-take-peer-review-as-the-gold-standard/2019/08/01/fd90749a-b229-11e9-8949-5f36ff92706e_story.html
- [1]Bruce Mazlish, The Uncertain Sciences (New Brunswick, NJ: Transaction, 2007).↩
- [2]Paul D. Thacker and Jon Tennant, “Why we shouldn’t take peer review as the ‘gold standard,’” Washington Post, August 1, 2019, https://www.washingtonpost.com/outlook/why-we-shouldnt-take-peer-review-as-the-gold-standard/2019/08/01/fd90749a-b229-11e9-8949-5f36ff92706e_story.html↩
- [3]Paul D. Thacker and Jon Tennant, “Why we shouldn’t take peer review as the ‘gold standard,’” Washington Post, August 1, 2019, https://www.washingtonpost.com/outlook/why-we-shouldnt-take-peer-review-as-the-gold-standard/2019/08/01/fd90749a-b229-11e9-8949-5f36ff92706e_story.html↩
- [4]Bruce Mazlish, The Uncertain Sciences (New Brunswick, NJ: Transaction, 2007).↩
- [5]Paul D. Thacker and Jon Tennant, “Why we shouldn’t take peer review as the ‘gold standard,’” Washington Post, August 1, 2019, https://www.washingtonpost.com/outlook/why-we-shouldnt-take-peer-review-as-the-gold-standard/2019/08/01/fd90749a-b229-11e9-8949-5f36ff92706e_story.html↩