Peer review of manuscripts and grants is part of the scientific endeavor and is an important responsibility of research scientists. However, how to do this critical task is rarely taught. Instead, many junior or early career scientists are handed a manuscript or invited to serve as a reviewer without much guidance. Like editors, reviewers serve as gatekeepers to the scientific literature. The primary role of the in-depth reviewer is to evaluate the technical and scientific soundness of the study. Some studies are very far from meeting the criteria needed for publication. Others need a few revisions to ensure that there is a sound basis for the conclusions and that the study is reliable and described adequately to be reproduced.
Journals generally provide instructions for reviewers either online at the journal or at the reviewer site or sometimes as one of the files provided with the manuscript. Most journals request an evaluation of the quality of the manuscript, an indication of any major or minor issues, and an impression of the impact of the study. How important impact is depends on the journal’s criteria for publication. Highly selective, high profile journals will use impact of the study as a key criterium, along with scientific rigor and effective presentation. Other journals may put less weight on this aspect of the evaluation and be more concerned with whether the study is appropriate for the specialized audience that the journal serves or whether the science presented meets the technical requirements of the journal.
The parts of a good review of a manuscript for publication include
- a disclosure of any limitations of the review
- a disclosure of any colleagues who assisted in the review (you should have received permission from the editor in advance)
- a disclosure of any potential biases or conflicts of interest
- a summary of the manuscript
- a numbered list of concerns
- a clear indication of the importance of each concern (major or minor, essential or optional)
- recommendations for how to address the concerns, if possible
What about manuscripts that do not have major issues? Providing a positive, but useful, review for a manuscript is less tricky than it sounds. The first four elements listed above still apply (limitations, colleague input, bias disclosure, and summary). After that point, what is helpful to an editor in making a decision is an indication of the importance of the study, a statement that you found the statistical analyses and n values sufficient, a statement that you found the experiments properly controlled and the experimental design appropriate and adequately described, a statement regarding if the cited literature is appropriate, and what kinds of future studies the manuscript will inspire. By including this information, the editor will be confident that you have critically evaluated the manuscript for scientific rigor and understood the experiments and data presented. You can also comment on the introduction and conclusions (or discussion), indicating if they are adequate or too long, and provide your opinion regarding the distribution of data between the main text and supplementary materials.
It is generally not appropriate to include in the comments to the authors your recommendation regarding publication (acceptance or rejection) of the manuscript. That information can be provided in the comments to the editor rather than in the comments to the authors. Ensure that, in your attempt to be diplomatic, you do not convey a different impression to the editor and the authors. If your comments for the editor are scathing and overwhelmingly negative or your ratings are low or your recommendation is rejection, be sure that the editor has sufficient information in the comments to the authors and that those comments are not phrased with complimentary language in the comments to the authors. If you state that the experimental design is poor and the experiments are poorly controlled and your recommend rejection in your comments to the editor, be sure that you are not positive and encouraging in your comments to the authors. Delineate the issues with the experimental design and describe the missing controls in your comments to the authors. Ensure that any negative recommendations or criticisms conveyed to the editor are supported in the comments to the authors. Be diplomatic, but accurate and precise about the concerns and issues. Avoid making critical but vague comments about the science, data, or methods.
If there is biased presentation of the existing literature, be sure to provide representative or key references so that the editor can properly evaluate this concern and convey it to the authors. This information can be presented in either the comments to the editor and to the authors. If there is a concern that disclosing the references will make the reviewer identifiable to the authors, then those references should be noted in the comments to the editor.
Avoid marking corrections on the manuscript file itself or performing detailed proofreading for grammar, spelling, or language. Those kinds of issues can be raised in a single sentence to the author, indicating that the manuscript contains typographical or language errors that should be addressed. One or two examples are fine, commenting on every instance is inappropriate. Instead, in the detailed comments to the authors note issues of data presentation or interpretation, flaws in logic, conclusions that are not supported by the data, biased citation of the literature, a lack of acknowledgment of opposing models or alternative interpretations, missing controls, poor experimental design, or inadequate explanation of methods.
When dividing issues into those that are major and those that are minor, consider these questions: Will addressing this concern change the interpretation or conclusions? Does this issue compromise reproducibility? If the answer is yes, then those are major concerns. If the answer is no, then it is likely the issues are minor. Minor issues include if the figures are consistently labeled, if the division of data between the main text and supplementary materials is appropriate, if the introduction adequately explains the purpose of the study and provides key background information, if the discussion is too long. Generally, issues that do not require new experiments or do not reflect bias in the study are minor. Those that require experiments or address bias or reproducibility of the data are major.
Not only is understanding the elements of peer review and how to provide an effective, competent, useful review critical for contributing to the scientific endeavor as an in-depth reviewer, but this knowledge will also help you draft more competitive manuscripts. Knowing the criteria for evaluation is key to writing a competitive manuscript. Being an effective reviewer is not easy. It is time consuming. However, helping other scientists improve their work and get closer to conveying their research to the world through a peer-reviewed publication is a tremendously rewarding part of being a scientist.
The video below is a segment from a career-training session. More information about career-training services
Related Resources
N. R. Gough, Training for peer review. Science Signaling 2, tr2 (2009). Full Text PubMed
M. B. Yaffe, Re-reviewing peer review. Science Signaling 2, eg11 (2009). Full Text PubMed
S. van Rooyen, N. Black, F. Godlee, Development of the review quality instrument (RQI) for assessing peer reviews of manuscripts. J. Clin. Epidemiol. 52, 625-629 (1999). PubMed
Cite as: N. R. Gough, Tips for Peer Review: A Guide to Providing a Useful Evaluation. BioSerendipity (10 September 2018) https://www.bioserendipity.com/tips-for-peer-review