Can peer review survive social science paradigm wars?
Peer review is essential to the functioning of research communities. But what are the requirements for it to work? One is certainly that there must be some minimum level of agreement on the task of research and how it should be pursued.
Yet increasingly, in many areas of the social sciences and humanities, there are fundamental divisions not just about what is studied and how to understand it, but even about what the product of research should be. For example: are human beings, organizations and institutions causal agents operating in the world? Where are they discursive constructions with nothing found “outside the text”? Does the research aim to understand the world or to have an “impact” on it, including by reducing social inequalities?
These are clashes between fundamental commitments, leading to the “paradigm wars” whose future Nate Gage predicted in 1989. Is peer review compatible with these conflicts? Let me illustrate the problem.
Recently, while reviewing an article for a newspaper, I was faced with a dilemma. The article raised some interesting points, but my view was that it was based on a series of questionable empirical, theoretical and political assumptions that led to biased interpretations of rather thin data. The authors suggested that any questioning of these assumptions amounted to an attack on their intellectual and social identity. But I felt that since many readers wouldn’t share them, an explicit rationale should be provided.
Given this, I recommended major revisions. Other reviewers were more supportive, although they identified various issues that needed attention. The editors decided on minor revisions. The resubmitted document did not make any substantive changes relevant to my comments. More importantly, the cover letter didn’t respond to me at all, only to the other two reviews.
The editors then asked the authors to respond to my comments, but their response was that, since they had a different “onto-epistemological position”, they did not need to offer counter-arguments to the specific points that I had raised. They claimed that the differences between us were “irreconcilable” and outside the scope of their article, and that my “view” informed my “personal opinion on ‘valid data'”. They insisted that their ‘methodologies’ had been ‘carefully endorsed by the academy as reliable, valid and trustworthy’. In other words, since they could draw on literature that shared their commitments, there was no need to justify them in the article.
I disagreed and recommended rejection, especially since, in my view, the document still had fundamental flaws. However, the other reviewer at this second stage recommended publication and commented, “Congratulations to these authors for returning to Reviewer 1.”
“Repel” sounds like a military metaphor; the implication seems to be that simply engaging with the critical points I had raised would be tantamount to surrendering to what needed to be pushed back. Note that what is at issue here is not that the authors did not edit their article; peer review does not require it. The problem is that they initially refused to answer and, when pressed, simply appealed to their own paradigm commitments. And it worked. The article will be published soon.
Of course, one could argue that I should have acknowledged the legitimacy of the alternative paradigm to which the authors claimed adherence and accepted that my critical comments were therefore unenforceable, merely reflecting my own paradigm. Many will say that it is a principle of university life that diversity of orientation should be welcomed, and I agree up to a point. But there are clearly limits to tolerance – indeed, both parties to the conflict discussed here have recognized this in practice. The question is: on what basis should these limits be determined?
The authors effectively rejected a key assumption underlying the peer review. They adopted a version of what has come to be called the epistemology of point of view, according to which certain points of view are made credible – and others discredited – according to the social category to which those who present them belong. . The views of members of marginalized or oppressed groups (or rather those who claim to speak on their behalf) are given epistemic privilege, while those held by members of what is considered the dominant group are rejected, by example as (at best) “opinions”.
Thus, on the one hand, the limits of what is acceptable were defined in terms of a paradigm whose authors freely acknowledged served political purposes; while on my side the limits have been taken to stem from what is needed for academic peer review to work. The conflict arises because peer review requires that all “peers” be treated equally, with none being epistemically privileged.
Indeed, anonymity is used to blind authors and reviewers (among other things) to each other’s social characteristics and political views, whenever possible. All that matters is that they are members of the same research community; and they are bound to engage with each other solely on that basis, not political or paradigmatic commitments.
Can dropping this requirement be tolerated in the context of peer review? It was never perfect, of course, but if peer review is just another vehicle for paradigm wars, can it still be justified?
Martyn Hammersley is Emeritus Professor of Educational and Social Research at the Open University.