Introduction
The future of peer review in scientific publishing should include artificial intelligence (AI). Large-language models, incorporating in-context learning, could assist editors in triaging manuscripts. Numerous questions have emerged over decades concerning peer review. Is it effective in detecting fabrication, falsification, image manipulation, or adherence to reporting guidelines? Can AI be fair, or subject to bias? Is it efficient, or increasingly labour-intensive without compensation? Given these challenges, AI should and will be increasingly used to assist in peer review.
Effectiveness of AI in Peer Review
Peer review is crucial for maintaining the quality of scientific publications. However, human reviewers often struggle to detect issues like data fabrication, falsification, and image manipulation. AI could be more effective at identifying these problems. For instance, large-language models could be trained to spot inconsistencies in data and adherence to reporting guidelines. A study by Liang et al. assessed the agreement between human reviewers and GPT-4 on 3096 manuscripts submitted to 15 Nature journals. They found a 30.1% overlap in comments, which is comparable to the 28.6% overlap between two human reviewers. This suggests that AI can provide valuable insights, potentially even more so than human reviewers.
Fairness in Peer Review
Bias in peer review is a well-documented problem. Various forms of peer review, such as open, single-blind, and double-blind, aim to mitigate bias. However, these methods are not foolproof. For example, a study found that manuscripts from authors with Nobel prizes were less likely to be rejected compared to those from unknown authors. AI could help reduce bias in peer review. Algorithms can be designed to evaluate manuscripts based solely on their content, without considering the authors’ identities or affiliations. This could lead to a fairer assessment process and reduce the influence of factors like language proficiency and institutional reputation.
Efficiency in Peer Review
The number of scientific publications has increased by 47% between 2016 and 2022, reaching 2.82 million manuscripts per year. This surge has overwhelmed editors and peer reviewers, leading to a higher rejection rate without external review. AI can help alleviate this burden by initially screening manuscripts and providing a summary of their quality. AI can also speed up the review process. For example, a recent study found that 57.4% of investigators in AI and computational biology found GPT-4’s feedback helpful, and 82.4% found it more helpful than some human reviewers. This indicates that AI can provide timely and valuable feedback, enhancing the efficiency of the review process.
The Future of AI in Peer Review
While AI shows promise, it is not yet ready to replace human reviewers entirely. Current systems need more data to improve their accuracy and reliability. However, AI can complement human reviewers by providing initial assessments and identifying potential issues. One concern is the confidentiality of manuscripts. Large-language models extract data from numerous published works, which could compromise the confidentiality of the reviewed manuscripts. A possible solution is to limit AI’s training data to the journal’s database or open-access articles.
In the future, AI could be used to scan all submissions and provide a quality summary for editors. This will help editors decide whether to proceed with external peer review. Embracing AI in peer review is inevitable, and the focus should be on evaluating its effectiveness and reassuring authors about the fairness of the process.
Conclusion
AI has the potential to address many challenges in the peer review process, including effectiveness, fairness, and efficiency. While it is not yet ready to replace human reviewers, it can significantly enhance the review process. As AI technology matures, it will likely play an increasingly important role in scientific publishing.
