Guidelines for reviewers
Note: a version-controlled version of these guidelines is maintained at https://github.com/review-commons
Table of Contents
- Writing the report
- Structure of the Report
- Referee Cross-commenting
- Conflict of interests
- Scientific integrity and ethical considerations
- Reviewing a revision for an affiliate journal
The primary purpose of the peer-review process at Review Commons is to provide an in-depth analysis of a scientific study in order to help the authors improve their paper by strengthening the rigor of the work and the clarity of the presentation.
Review Commons also requests reviewers to assess how the study advances the field rather than establishing its fit for a particular journal. The reviews should be scholarly in tone and content will be made public (by default without referee names) unless the authors opt out. As such, reviews should also be useful to all potential readers to better understand the work and its context and to journal editors to make informed decisions.
Writing the report
Review Commons referee reports are expected to be scholarly communications intended to advance scientific discourse. Authors can make reviews public, either at the stage of author-requested posting of a Refereed Preprint (the term for the paper, reviews, and author response posted on bioRxiv) or at the stage of journal publication. Please provide a clear, professional and impartial analysis that you would employ if you were signing the review, even if you are not (signing of reports is encouraged but not required by Review Commons). Be mindful of potential sources of bias with regard to the authors, including gender, ethnicity, geographical location, seniority, reputation in your field of expertise. The Managing Editor has the right to exclude reviews that are deemed unprofessional in tone or content.
Hallmarks of a good referee report are:
- Timely delivery
- Critical and impartial analysis with substantiated arguments.
- Constructive and realistic suggestions.
Review Commons believes that early career researchers (ECRs) can play a valuable role in reviewing, provided that the principal reviewer has carefully read the paper and provides careful mentorship of the review process. ECRs are defined here as advanced graduate students or postdocs with prior experience in publishing either as an author or reviewer. Transparency of all participating reviewers is an essential aspect of Review Commons. We expect the principal reviewer to identify ECRs who are well-qualified to participate in the review process and to be responsible for the quality of the final review. The review form includes a field to list the names of all co-reviewers.
Structure of the Report
Review Commons recommends including the following components in referee reports:
1. Evidence, reproducibility and clarity
Provide a short summary of the findings and key conclusions (including methodology and model system(s) where appropriate). Please place your comments about significance in section 2.
- Are the key conclusions convincing?
- Should the authors qualify some of their claims as preliminary or speculative, or remove them altogether?
- Would additional experiments be essential to support the claims of the paper? Request additional experiments only where necessary for the paper as it is, and do not ask authors to open new lines of experimentation.
- Are the suggested experiments realistic in terms of time and resources? It would help if you could add an estimated cost and time investment for substantial experiments.
- Are the data and the methods presented in such a way that they can be reproduced?
- Are the experiments adequately replicated and statistical analysis adequate?
- Specific experimental issues that are easily addressable.
- Are prior studies referenced appropriately?
- Are the text and figures clear and accurate?
- Do you have suggestions that would help the authors improve the presentation of their data and conclusions?
- Describe the nature and significance of the advance (e.g. conceptual, technical, clinical) for the field.
- Place the work in the context of the existing literature (provide references, where appropriate).
- State what audience might be interested in and influenced by the reported findings.
- Define your field of expertise with a few keywords to help the authors contextualize your point of view. Indicate if there are any parts of the paper that you do not have sufficient expertise to evaluate.
Review Commons referee reports do not include ‘confidential comments to the editors’. In case of potential ethical breaches or data integrity issues, please contact the editors directly.
Before sending the reviews back to the authors, Review Commons will circulate the reports to all referees with an invitation to comment on each others’ reports where appropriate. These comments will be included in the full reports forwarded to the authors where appropriate.
By default, the identity of the referees will not be communicated to the authors, unless a reviewer explicitly decides to sign their report, which they are welcome to do.
All referees must agree to disclose their names to any affiliate journals and to have their comments posted on a preprint server and/or as part of the transparent review process at a Review Commons affiliated journal. Referee names will only be posted if a referee elects to sign their reports.
Some of the affiliate journals implement referee cross-commenting or consultative procedures (e.g. eLife) which require the reviewers to share their identity with the other reviewers (but not with the authors).
All Review Commons referees are asked to agree to participate in principle to such consultation at affiliate journal when necessary.
Conflicts of interest
Review Commons will avoid inviting referees who have recent or ongoing collaborations with the authors, are or will be engaged in similar research, have commented on drafts of the manuscript, are in direct competition, work in the same department or institution, have a personal relationship, have a history of dispute with the authors, or have a financial or personal interest in the outcome. Because it is not possible for the editors to know of all possible biases, however, we ask referees to draw our attention to anything that might affect their report, including commercial interests, and to decline to referee in cases where they feel unable to be objective and impartial. Previous review of a version of the submitted study elsewhere does not a priori exclude re-review by the same referees, unless they feel predisposed against the work.
Referees should maintain strict confidentiality on all aspects of the review process and keep the following guidelines in mind:
Manuscripts reviewed for Review Commons that are not yet publicly available as preprint should not be discussed with anyone not directly involved in the review process. If experts from outside the referee’s own laboratory are consulted, referees should check with the editors beforehand to avoid involving anyone who may have been excluded by the editor.
Scientific integrity and ethical considerations
In spite of our best efforts to identify breaches of publication policy or ethical conduct, such as plagiarism or author conflict of interest, the referees who are more familiar with the field are more likely to recognize such problems and should alert the editors to any potential problems in this regard. If preferred, this can be done outside the formal peer review form.
Review Commons reserves the right to reject manuscripts at any point if ethical, biosecurity, or scientific integrity issues arise.
Reviewing a revision for an affiliate journal
One of the major goals of Review Commons is to improve the efficiency of the peer-review process by avoiding multiple cycles of re-review by different sets of reviewers. When authors decide to submit their reviewed manuscript to one of the journals affiliated with Review Commons, it is essential that the journal can contact the original reviewers to review a revised version of the study where appropriate. Affiliated journals are encouraged to only contact referees where this is required for an informed editorial process (such as for clarification or to evaluate data added in revision), and not to request an editorial opinion on journal fit.
When reviewing a revised version of a manuscript, it is important to focus the evaluation on the points raised in the initial reviews, including those by the other reviewers, and to avoid raising new issues unless fatal flaws are discovered. If new issues arise from data added during revision, changes should be requested only if absolutely necessary and directly related to the major conclusions of the study.
If authors decide to take their manuscript to a journal outside of the affiliate journals, the editors may approach Review Commons for the name of reviewers. In such cases, Review Commons will always ask reviewers for permission before forwarding their name to a journal not affiliated with the consortium. The referees are free to decide whether to re-review a revised manuscript for a journal that is not affiliated with the platform.