
Are peer reviews a waste of time? | thearticle
- Select a language for the TTS:
- UK English Female
- UK English Male
- US English Female
- US English Male
- Australian Female
- Australian Male
- Language selected: (auto detect) - EN
Play all audios:

The report of the Commission on Race and Ethnic Disparities, which was published on 31 March, was greeted by an avalanche of hostile commentary. Precisely how seriously we should take the
criticisms is debatable. Iain Dale, an informed commentator, noted, “The number of people who castigated and slammed it within minutes of its release can’t possibly have read all of its 258
pages.” He concluded, “They just jerked their knees in the time-honoured fashion. Basically, the criticism was based on the fact that Boris Johnson had commissioned it, ergo it must be
biased, useless or bad, or all three.” Very unpleasant _ad hominem_ attacks, such as that by the Cambridge professor, Priyamavada Gopal, did little to raise the tone of debate. One of the
more rationally-argued criticisms, on the website of the _British Medical Journal_, noted that, “The 30 page section on health in the report claims to undo several decades of _irrefutable
peer reviewed_ research evidence on ethnic disparities…” (Emphasis added.) I aim to explore the use of peer review, particularly in the social sciences. Its connection to contentious areas
of research, like racism, has had an eventful recent history. Peer review is one of the cornerstones of academic research. It means that an article which has been submitted for publication
has to be reviewed by experts in the field, to ensure that the article’s arguments are robust and its conclusions reliable. Peer review is generally regarded as the gold standard in academic
research quality. It is a system which has been tried and tested for centuries. It works. However, it is not foolproof. The medical journal _The Lancet_, for instance, uses peer review
rigorously. Even so, it still managed to publish Andrew Wakefield’s now notorious and debunked article in 1998, which linked autism with the MMR vaccine. Sadly, this was not an isolated
example. The frauds listed here did not all involve the defeat of the peer review system, but many did. Moreover, this list covers just the hard sciences, where ideology and politics are
absent, and where input from postmodernist theory is minimal. It is a very different picture in the social sciences and humanities. Peer review is excellent at catching individuals or small
groups who “turn rogue”. But what happens if an entire field of research, perhaps a field which came into existence very recently, is colonised by people who share the same ideological or
political convictions? We can surmise that peer review would become useless. It would merely ask people of the same ideological world view to mark each other’s homework. Worse, it would
become a mechanism to silence those who do not share the True Faith. Papers which fundamentally challenge the accepted orthodoxy would never see light of day in the core periodicals of the
field. All this is, of course, speculation. Is it borne out by reality? The academic rigour that peer review implies was always assumed, but until recently no one thought to test it. Then,
in 1994, an American physicist called Alan Sokal decided to see how easy it would be to perpetuate an academic fraud. He submitted an article to the Duke University periodical _Social Text_.
We might fairly describe it as a periodical in the postmodernist _avant garde_. In his article, Sokal used postmodernist ideas to challenge the fundamental basis of science. To put it
slightly crudely, he argued that something like the theory of gravity is merely a figment of the imagination of the colonialist, white supremacist patriarchy, and it is therefore nothing
more than a tool to keep the oppressed masses in their place, on _terra firma_. (After all, it was first put forward by a white, English-speaking male. That alone has to be grounds for
suspicion.) Sokal quoted all the right theorists and used all the right jargon to make it look as though he knew what he was talking about, but took care to include enough deliberate errors
in basic science to make it clear that his article was rubbish. _Social Text_ published Sokal’s article. When he revealed the hoax, his fellow scientists enjoyed the joke. Social scientists
were rather less amused. However, _Social Text_ was not peer reviewed (although it is now). This is why a trio of scholars, most notably Peter Boghossian, decided to conduct a larger-scale
test in 2017-18, but aimed purely at peer reviewed periodicals. They, like Sokal, had doubts about the quality of scholarship in the social sciences and humanities. They chose the most
politically contentious areas of social science research, such as race, gender and sexuality, which they dubbed “grievance studies”. Boghossian _et al_ set themselves the goal of getting
twenty fictitious papers published in the most prestigious, peer reviewed periodicals in their fields. To do this, they had to produce papers which would not raise suspicion in the mindsets
of the people who edited and reviewed papers for these periodicals, but which would be obviously bogus to the general public. To that end, they studied the papers these periodicals were
publishing, until they could mimic the style word-perfectly. Then they pushed the envelope slightly, to come up with ideas for their own papers that were either absurd or unethical. Their
first attempt, which argued that the male penis conceptually causes climate change, was not a success. It was too obviously a hoax. Later attempts were much more successful. Papers which
were accepted for publication argued: that society’s prejudice against unhealthy, obese bodies was an example of oppression; that men could be cured of homophobia and transphobia by
encouraging them to masturbate anally with dildos; that men who visited restaurants like “Hooters” to gaze at waitresses with big breasts were examples of the oppressive patriarchy; and that
social justice scholarship should be immune to criticism. They managed to get a feminist reworking of a chapter from Hitler’s _Mein Kampf _accepted for publication, and even a rambling,
misandrist poem, which was written by a random poetry generator on the internet. One of their papers, which was still under consideration when the experiment was ended prematurely, argued
that artificial intelligence is too male-oriented, in that it is rationalist and masculinist, and that it needs to be female-oriented. It is noteworthy that another paper, written by
academics at New York University, and which argues almost exactly the same thing, was published in May 2019. One fears the NYU academics were being entirely serious. The _pièce de
résistance_ was a paper on dogs which rape other dogs in the park. It argued that the lessons learned here could be used to train men to refrain from sexual violence and misogyny. Not only
was this accepted for publication, it was acclaimed for its outstanding scholarship. (The underlying aim was to see if a paper whose argument was ludicrous and unfalsifiable would be
accepted. Clearly, it was.) It was only when journalists took an interest, and wondered why neither the purported author nor the academic institution which employed her existed, that
Boghossian _et al_ came clean and admitted what they were doing. That ended prematurely the experiment that has become known as “Sokal Squared.” The final tally was: seven papers accepted
for publication, six rejected, and the remaining seven still under consideration for publication. The ramifications of Sokal and Sokal Squared will reverberate in academia for some time to
come. In the meantime, politicians and civil servants, who rely on academic research for a secure platform for formulating public policy, have some thinking to do. Just how reliable is the
peer reviewed research in ideology-driven fields, and can we take it on someone’s say-so that it is “irrefutable”? I am not suggesting that any specific piece of research, or whole fields of
research have been corrupted by ideological conviction. On the contrary, I am arguing that we cannot assume they have not – even when they have been subjected to the purity test of peer
review. A MESSAGE FROM THEARTICLE _We are the only publication that’s committed to covering every angle. We have an important contribution to make, one that’s needed now more than ever, and
we need your help to continue publishing throughout the pandemic. So please, make a donation._