How might we improve the peer review experience?
The Publisherspeak peer review survey 2022 results

In 2022, we wanted to take a closer look at one of the pillars that are central to the scholarly research process: peer review. Our goal with the survey, How might we improve the peer review experience? was to understand the unique perspectives and challenges of authors, reviewers, editors, and publishing team members. In this survey, we looked at peer review from two angles: one, the challenges faced by different stakeholders, and two, the gaps in the current processes.

Now, the stakeholders in peer review have spoken, and we are excited to share with you the results of the survey and the insights we gained from different participants in the peer review process.

Survey methodology and background information

We published our peer review survey in March 2022. The survey included questions about the role(s) played by the recipient in the peer review process, their typical challenges, the improvements they would like to see in the peer review process and platforms, and more. This survey was shared via email campaigns and posts on social media. 

People playing different roles in the peer review process participated in this survey over a period of roughly 3 weeks. 34% of respondents were from Europe, 24% from North America, 24% from South America, and the remaining 18% from Asia.

Authors made up 16% of the respondents, while 17% were reviewers, 28% were editors, and 39% were publishing team members. Several respondents chose multiple roles, with 7.7% of respondents choosing all four roles – author, reviewer, editor, and publishing team member.

The authors’ perspective

Starting with the central figure of the publishing process, we asked authors what their typical challenges and frustrations were while submitting manuscripts. The 3 most popular responses were having to use multiple tools to get their manuscripts ready for submission, understanding and adhering to submission guidelines, and ensuring that their manuscript language quality is up to the mark.

75% of the respondents that chose manuscript language quality as one of the challenges were from Europe and Asia. This demonstrates the importance of multilingual support in journal publishing to make the ecosystem more equitable and accessible to non-English speaking authors and readers.

Other notable challenges include ensuring that the author’s critical data and assets are validated properly and having to arrange references as per journal guidelines.

When asked about the improvements that they would like to see in the peer review process, here are the top 3 improvements the authors picked:

  1. Better visibility & updates of the review process
  2. Ease of resubmission of revised manuscript
  3. Structured and clear feedback reports and revision requests

A few authors also mentioned that they would like to have a smaller time gap between submission and peer review and the ability to manage all their submissions in one place.

The reviewers’ POV

Reviewers are critical stakeholders in the peer review process; they help uphold the integrity of published research as external referees with their voluntary services and contributions. According to the peer reviewers that took our survey, the top problems they face during the peer review process include:

  1. Confirming whether manuscript revisions have addressed their review comments
  2. Giving structured, contextual, and constructive feedback
  3. The lack of quality in data, assets, and language in submitted manuscripts

A few reviewers also highlighted issues with accessing and using peer review platforms, managing and tracking multiple manuscripts for review, and the lack of clarity in review guidelines and parameters.

We also asked reviewers what would be the one critical factor that would improve the quality of their peer review experience. Reviewers felt that validation of submission data, assertions in the manuscript, references, etc., structured feedback forms and templates to add review comments, and ease of use of the peer review system are key to a successful peer review process.

While structured feedback forms and templates were one of the top choices of reviewers, a small minority shared a different perspective and argued that it may be better not to have any feedback forms as some forms provided by the journals cover a multitude of unimportant issues, which detracts attention away from important issues. 

The editors’ experience

Editors help guide the peer review process and ensure that it is fair, timely, and rigorous. We asked editors about the challenges they face during peer review, and here are their top responses:

  1. Finding suitable peer reviewers
  2. Managing their peer review workload – assigning reviewers, tracking, and following up on review status
  3. The lack of proper reports

Other popular responses include ensuring a fair and unbiased review process, capturing and sharing reviewers’ feedback in an organized manner, and the lack of configurability in setting up different peer review workflows. A few editors also revealed that it is often difficult to ensure that reviews are completed on time and deadlines are met because reviewers are often busy with their own research.

On the subject of peer review models, we asked editors which model they foresee as being the most beneficial – single-blinded, double-blinded, transferable, collaborative, post-publication, open peer review, or any other model(s). 

69% of editors chose double-blinded peer review as the most beneficial model. When asked why, the main reasons that emerged were:

  1. Double-blinded review gives the researcher the freedom to review papers even from their close colleagues, making observations that they may not do in person. So it is more fair.
  2. Double-blinded review allows for less bias.
  3. While the trend in other areas is to make everything collaborative and open, there is a lot of value in not knowing names and keeping this in reserve to avoid “bad practices.”

The second most popular option, chosen by 19% of respondents, was open peer review, followed by collaborative and single-blinded peer review (6% each). 

The publishing teams’ outlook

Publishing teams play the role of overseeing the peer review process and enabling the smooth dissemination of research. A peer review system is a critical tool that enables publishing teams to do so. Here are the most popular factors publishing team members consider while making the decision to adopt a peer review system:

  1. Affordable pricing (eg. pay-per-use pricing, low setup fees, etc.)
  2. Seamless integration with industry tools and databases
  3. Seamless integration with production systems
  4. Flexible blinding options and customizable workflows

Other notable factors include plagiarism detection, actionable reports, adoption of AI/ML and blockchain technology, and multilingual support.

Here are the most common challenges that publishing team members said they face during the peer review process:

  1. Finding suitable peer reviewers
  2. Having visibility over and managing the peer review workflow
  3. The lack of configurable workflows (eg. blinding, open peer review, etc.)
  4. The ability to seamlessly cascade to other journals
  5. Portability of metadata into production systems after acceptance
  6. Capturing and sharing reviewers’ feedback in an organized manner

Publishing team members also mentioned challenges around long review cycles, duplicate submissions, and the lack of integrations of peer review platforms with third-party systems like ORCID, Crossref Funder Registry, etc.

Peer review is essential to research; this process plays the critical role of upholding research integrity. It has come a long way in the last few decades, but there is much scope for improving the process to make it a smoother and more rewarding experience for everyone involved. This calls for the research community to collaborate as a cohesive unit for better execution of peer review that will ultimately help humanity address the challenges we face.