Peer Review Week 2018 is all about diversity. To look at the changing scholarly landscape, we interviewed science publishing historian, Aileen Fyfe, who draws on her experience working in the archives of the Royal Society to explain how diversifying the pool of experts involved in the peer review process can further advance the scientific record.
How diverse is the peer review process at the moment?
Until more people start publishing their data on both their community of authors and reviewers it’s difficult to tell. There’s certainly concerns about the geopolitical balance, whether the global north is doing much more of the refereeing, whereas the global south is doing a lot of authoring, and not so much refereeing. So that’s a large scale problem. There are certainly suspicions about women’s involvements, whether women are underrepresented as reviewers and editorial board members. The stats that we have so far do suggest that that’s true. But we still don’t know very much about other forms of diversity.
One of the points that I’d like to make about diversity in peer review is that there is always going to be some limit on how diverse peer review can be, because peer review by its nature doesn’t let just anyone do it. So we select as peer reviewers people who have got some form of shared expertise, some certain standards of training and education, and some form of research credentials, which nowadays generally means having published a couple of papers. So, there’s already a limit on exactly how diverse the pool can be, there are going to be some people who are not going to be invited to do this. And for peer review to work we wouldn’t want to lose that. But the interesting question becomes, within those limits, within the pool of people who have the appropriate level of education and professional research training are there any other reasons why diversity should be restricted. Nowadays we would probably say no to that, we wouldn’t think that religion or gender, geographical location or socioeconomic status should have anything to do with it. But those things have all often historically been tied up.
Over the years we’ve extended our understanding of who can be involved in peer review, but we don’t want to lose the sense of recognising what good science looks like. But, on the other other hand, there is room for a lot within that.
What happens when the peer review process isn’t diverse?
An immediate worry is that the process wouldn’t be fair if, for instance, decisions about global science were routinely being made by a handful of elite researchers in a handful of elite institutions. And we know that everyone carries implicit biases against people who are not like them—which is a particular worry in those fields of research that currently use single blind review systems.
But, you’ve also got the worry that with any group of similar minded people who share an understanding of what good science looks like, you’ve got a tendency to commit towards intellectual conservatism. Will they recognise the innovation, the speculative approach that turns out to be really fruitful? Because if you’re looking for stuff that is familiar and solid and rigorous and good, maybe you’ll miss something that’s exciting and novel, so there’s also that kind of aspect of diversity.
This intellectual diversity is slightly different from the social economic, gender, class, ethnicity that we talk about today. And nowadays we try to separate those things, and reckon hopefully we can have intellectual diversity and also have diversity of types of people.
We want to try and ensure that the people doing the editorial and reviewing work, aren’t being too prone to groupthink if you will, that they won’t be dismissing new approaches, new ideas, because they don’t look like the ones that they are familiar with, so there’s a worry that the scientific content could suffer if all the people doing the reviewing all comes from the same institution or from the same sub-field of a discipline.
What can we do to support diversity in the peer review process and scientific discourse in general?
One of the things that does seem to work is talking about it! When I look at the change in statistics for the Royal Society’s journals over the last 30 years, it’s quite clear that something has happened…, in terms of who is writing for the Royal Society and reviewing for the Royal Society, that has changed a lot since the 1980s early 1990s and I think it’s changed much more recently than that, and you have to ask why. And the only reason that I can see is that that organisation has started talking a lot about diversity, it has started gathering the statistics, and it’s started wondering what it can do better. And it looks as if that works. In terms of the number of editors, this is still rather male only, but the editorial boards, the publicly visible aspect, is getting better; and diversity among reviewers themselves is also getting better, it’s now similar to the number of authors in terms of gender. The proportion of women who are reviewers is similar to the proportion of women who are authors, which seems like a reasonable place to be.
So, talking about it, asking questions, getting people to publish their stats.
I think that that raising awareness thing, which is becoming more common, does seem to be focusing people’s minds, even if it’s just focusing editors’ minds… on who are they going to choose. And that’s the first step towards making some sort of progress, so they don’t automatically go to whatever the equivalent of an old boys club is right now…. And this could be your institutional network, the people you know and went to graduate school with, it could be that kind of group, and ideally you want something more diverse than that. And if you got the editors thinking about that, then that’s got to be a good first step.
About Aileen Fyfe
Aileen is a professor of history at the University of St. Andrews where she researches the history of scholarly communications. She currently leads a research project based at the Royal Society investigating the history of the world’s oldest scientific journal, Philosophical Transactions.
Featured image: Figure 1, PONE 0197280