Are peer reviews a waste of time? | thearticle

Are peer reviews a waste of time? | thearticle


Play all audios:


The report of the Commission on Race and Ethnic Disparities, which was published on 31 March, was greeted by an avalanche of hostile commentary. Precisely how seriously we should take the


criticisms is debatable. Iain Dale, an informed commentator, noted, “The number of people who castigated and slammed it within minutes of its release can’t possibly have read all of its 258


pages.” He concluded, “They just jerked their knees in the time-honoured fashion. Basically, the criticism was based on the fact that Boris Johnson had commissioned it, ergo it must be


biased, useless or bad, or all three.” Very unpleasant _ad hominem_ attacks, such as that by the Cambridge professor, Priyamavada Gopal, did little to raise the tone of debate. One of the


more rationally-argued criticisms, on the website of the _British Medical Journal_, noted that, “The 30 page section on health in the report claims to undo several decades of _irrefutable


peer reviewed_ research evidence on ethnic disparities…” (Emphasis added.) I aim to explore the use of peer review, particularly in the social sciences. Its connection to contentious areas


of research, like racism, has had an eventful recent history. Peer review is one of the cornerstones of academic research. It means that an article which has been submitted for publication


has to be reviewed by experts in the field, to ensure that the article’s arguments are robust and its conclusions reliable. Peer review is generally regarded as the gold standard in academic


research quality. It is a system which has been tried and tested for centuries. It works.  However, it is not foolproof. The medical journal _The Lancet_, for instance, uses peer review


rigorously. Even so, it still managed to publish Andrew Wakefield’s now notorious and debunked article in 1998, which linked autism with the MMR vaccine. Sadly, this was not an isolated


example. The frauds listed here did not all involve the defeat of the peer review system, but many did. Moreover, this list covers just the hard sciences, where ideology and politics are


absent, and where input from postmodernist theory is minimal. It is a very different picture in the social sciences and humanities. Peer review is excellent at catching individuals or small


groups who “turn rogue”. But what happens if an entire field of research, perhaps a field which came into existence very recently, is colonised by people who share the same ideological or


political convictions? We can surmise that peer review would become useless. It would merely ask people of the same ideological world view to mark each other’s homework. Worse, it would


become a mechanism to silence those who do not share the True Faith. Papers which fundamentally challenge the accepted orthodoxy would never see light of day in the core periodicals of the


field. All this is, of course, speculation. Is it borne out by reality? The academic rigour that peer review implies was always assumed, but until recently no one thought to test it. Then,


in 1994, an American physicist called Alan Sokal decided to see how easy it would be to perpetuate an academic fraud. He submitted an article to the Duke University periodical _Social Text_.


We might fairly describe it as a periodical in the postmodernist _avant garde_. In his article, Sokal used postmodernist ideas to challenge the fundamental basis of science. To put it


slightly crudely, he argued that something like the theory of gravity is merely a figment of the imagination of the colonialist, white supremacist patriarchy, and it is therefore nothing


more than a tool to keep the oppressed masses in their place, on _terra firma_. (After all, it was first put forward by a white, English-speaking male. That alone has to be grounds for


suspicion.) Sokal quoted all the right theorists and used all the right jargon to make it look as though he knew what he was talking about, but took care to include enough deliberate errors


in basic science to make it clear that his article was rubbish. _Social Text_ published Sokal’s article. When he revealed the hoax, his fellow scientists enjoyed the joke. Social scientists


were rather less amused. However, _Social Text_ was not peer reviewed (although it is now). This is why a trio of scholars, most notably Peter Boghossian, decided to conduct a larger-scale


test in 2017-18, but aimed purely at peer reviewed periodicals. They, like Sokal, had doubts about the quality of scholarship in the social sciences and humanities. They chose the most


politically contentious areas of social science research, such as race, gender and sexuality, which they dubbed “grievance studies”.  Boghossian _et al_ set themselves the goal of getting


twenty fictitious papers published in the most prestigious, peer reviewed periodicals in their fields. To do this, they had to produce papers which would not raise suspicion in the mindsets


of the people who edited and reviewed papers for these periodicals, but which would be obviously bogus to the general public. To that end, they studied the papers these periodicals were


publishing, until they could mimic the style word-perfectly. Then they pushed the envelope slightly, to come up with ideas for their own papers that were either absurd or unethical. Their


first attempt, which argued that the male penis conceptually causes climate change, was not a success. It was too obviously a hoax. Later attempts were much more successful. Papers which


were accepted for publication argued: that society’s prejudice against unhealthy, obese bodies was an example of oppression; that men could be cured of homophobia and transphobia by


encouraging them to masturbate anally with dildos; that men who visited restaurants like “Hooters” to gaze at waitresses with big breasts were examples of the oppressive patriarchy; and that


social justice scholarship should be immune to criticism. They managed to get a feminist reworking of a chapter from Hitler’s _Mein Kampf _accepted for publication, and even a rambling,


misandrist poem, which was written by a random poetry generator on the internet.  One of their papers, which was still under consideration when the experiment was ended prematurely, argued


that artificial intelligence is too male-oriented, in that it is rationalist and masculinist, and that it needs to be female-oriented. It is noteworthy that another paper, written by


academics at New York University, and which argues almost exactly the same thing, was published in May 2019. One fears the NYU academics were being entirely serious. The _pièce de


résistance_ was a paper on dogs which rape other dogs in the park. It argued that the lessons learned here could be used to train men to refrain from sexual violence and misogyny. Not only


was this accepted for publication, it was acclaimed for its outstanding scholarship. (The underlying aim was to see if a paper whose argument was ludicrous and unfalsifiable would be


accepted. Clearly, it was.) It was only when journalists took an interest, and wondered why neither the purported author nor the academic institution which employed her existed, that


Boghossian _et al_ came clean and admitted what they were doing. That ended prematurely the experiment that has become known as “Sokal Squared.” The final tally was: seven papers accepted


for publication, six rejected, and the remaining seven still under consideration for publication. The ramifications of Sokal and Sokal Squared will reverberate in academia for some time to


come. In the meantime, politicians and civil servants, who rely on academic research for a secure platform for formulating public policy, have some thinking to do. Just how reliable is the


peer reviewed research in ideology-driven fields, and can we take it on someone’s say-so that it is “irrefutable”? I am not suggesting that any specific piece of research, or whole fields of


research have been corrupted by ideological conviction. On the contrary, I am arguing that we cannot assume they have not – even when they have been subjected to the purity test of peer


review. A MESSAGE FROM THEARTICLE _We are the only publication that’s committed to covering every angle. We have an important contribution to make, one that’s needed now more than ever, and


we need your help to continue publishing throughout the pandemic. So please, make a donation._