JMIR Publications Knowledge Base and Help Center

Articles in this section

See more

(for reviewers) How to write a high-quality peer review

Avatar
JMIR Editorial Team
  • Updated
Follow

Background

A high-quality peer review can provide authors with thoughtful and respectful criticisms of the work, with the aims of improving the quality of the manuscript and its potential contributions to existing scientific knowledge. Additionally, such a peer review encourages authors to produce high-quality research and can help maintain scientific integrity and authenticity (including validity, significance, and originality of a study).

This article is intended to provide general guidance and suggestions to peer reviewers on what JMIR Publications considers to be a high-quality peer review. A high-quality peer review report on a scientific paper typically contains several key components that help evaluate the quality and validity of the research. 

Content:

Am I able to provide a high-quality peer review?

In alignment with high-quality publication ethical practices, peer reviewers should determine whether they are able to provide peer review comments on a manuscript. For example, peer reviewers can consider:

  • You are a scientific expert or have specialized knowledge about the content of a manuscript.
  • You can provide a constructive peer review (e.g., assess the validity of the science and methodological rigor, significance and originality of the research, contributions to scientific advancement; identify scientific errors and missing/incorrect references).
  • You can provide a peer review using statements of fact and neutral language. 
  • You have no conflicts of interest with the authors of the manuscript. (Review: How does JMIR define a Conflict of Interest (COI)?)
  • You can submit a peer review by the requested due date.

It is important for reviewers to adhere to ethical guidelines and maintain a constructive and professional tone throughout the review process. Providing specific and helpful feedback is crucial to ensuring the quality and integrity of the peer review system.

Anatomy of a high-quality peer review

While this is an extensive list, peer reviewers may not necessarily be required or expected to address every point for any given article type that is under review. However, many of these principles are best practices, and the editor may ask peer reviewers to clarify elements that they may deem important or vital to the peer review of a manuscript.

Here are some important elements commonly found in a comprehensive peer review report:

  1. Summary or Overview: (optional) Begin the report with a concise summary of the paper's main findings, objectives, significance, strengths and limitations. This helps provide an overview for the editor and authors.
  2. For Original Papers and Literature Reviews: (What are the article types for JMIR journals?)
    • Abstract: For structured abstracts, is the content of each section sufficiently detailed enough to understand the complete work within the 450-word limit?
    • Introduction: Evaluate the clarity and coherence of the introduction, including the presentation of relevant background information, the research question (or aims, hypotheses) and the significance of the study.
      • Literature overview and references: Evaluate the authors' knowledge of relevant prior research and their appropriate citation and referencing of previous studies. Identify any major gaps or omissions in the literature review.
    • Methods: Assess the methodology used in the research, including the study design, data collection methods, sample size, statistical analyses, and any potential limitations or biases. Does the manuscript reporting adhere to a relevant reporting guideline for the methods used? (What reporting guidelines should I follow for my article?) Are Ethical Considerations or Approvals included? ((for authors) Institutional Research Board / Research Ethics Board and Informed Consent )
    • Results: Evaluate the presentation and accuracy of the results. Check if they align with the research question, whether the statistical analyses are appropriate, and if the interpretation of the results is logical and supported by the data.
    • Discussion: Do the authors cover Principal Findings (and Strengths), Comparison to Prior Work, Limitations, and Conclusion? Assess the coherence of the discussion, the authors' interpretation of the results, and their ability to draw meaningful conclusions. Identify any unsupported claims, overgeneralizations, or opinions (e.g., recommendations, lessons learned, or otherwise) that are not based upon empirical findings or cited existing literature. Check whether the limitations of the study are properly acknowledged in the discussion section. All statements in the Discussion/Conclusions appropriately match the conclusions that can be drawn, i.e. ensure that conclusions can be drawn based on the methodology and findings of the study. (This applies to both the Abstract and the manuscript text.) 
  3. Clarity, organization, readability: Assess the clarity, organization, and overall flow of the paper. Identify any sections that need improvement or restructuring. Comment on the use of headings, subheadings, and other structural elements.
    • For Viewpoint manuscripts, these points can be especially important to clearly state the opinion or perspective taken by the authors, including key messages and supporting evidence for the argumentation included. If the Viewpoint instead describes an experience or lessons learned, the key aim and messages should be clearly stated and supported by specific examples from the experiences or observations of of the authors. Sufficient citation of existing scientific evidence would still be important to help place such experiences in context. 
  4. Language and style: Although peer reviewers may evaluate the quality of writing, including grammar, spelling, and sentence structure, this should not be the sole focus of the peer review report. Provide suggestions for improvement if necessary, and specifically in instances where the language quality interferes with the clear communication of the scientific work. Regarding style, ensure that the language is concise and accessible to the intended audience. JMIR Publications offers an Extended Copyediting Service when needed after a manuscript is accepted to help refine the English language. 
  5. Ethical Considerations: Identify any potential ethical concerns, such as plagiarism, conflicts of interest, or issues related to human or animal subjects. Comment on the ethical soundness and compliance of the study.
  6. Recommendations: Offer constructive feedback and specific recommendations for improvement. Highlight any major revisions that should be made, and suggest additional experiments or analyses if needed.
  7. Confidential Comments: Peer reviewers may provide optional confidential comments to the editor. 

Important: Although some peer reviewers may opt to use available generative artificial intelligence tools (E.g., GPT-4, etc) to assist in their process of peer reviewing manuscripts, we expect peer reviewers to disclose usage of such tools in their peer review reports. This is similar to the JMIR Publications policies for authors (How should the "Acknowledgments" section be formatted?) because ultimately, the peer reviewer is accountable for the submitted peer review report and comments.

Common pitfalls to avoid

  1. Lack of specificity: A poor-quality review report often lacks specific and detailed comments. It may contain vague statements or generalizations without providing specific examples or evidence to support the reviewer's critique or suggestions. On the other hand, excessive focus on copyediting related comments is not recommended.
    • Failure to Consider the Paper's Context: A subpar review may not take into account the specific aims, scope, or target audience of the journal or conference for which the paper is submitted. It may overlook the relevance or significance of the research within the broader scientific community.
    • Inadequate assessment: The reviewer may fail to critically evaluate the research methodology, results, or conclusions. They may overlook important flaws, errors, or inconsistencies in the study, or fail to identify gaps in the literature or limitations of the research. If the peer reviewer does not have the expertise to assess certain components of the study, this is acceptable and should be disclosed in Confidential Comments to the Editor.
    • Insufficient evaluation of ethical considerations: A poor-quality review report may overlook potential ethical issues, such as plagiarism, conflicts of interest, or inadequate protection of human subjects. Ethical concerns should be carefully evaluated and addressed in a review report. (for authors) Institutional Research Board / Research Ethics Board and Informed Consent)
    • Inconsistent or incoherent feedback: The feedback provided in a subpar review report may be inconsistent, contradictory, or incoherent. The reviewer may jump between different topics without maintaining a logical flow or fail to provide a clear overall assessment.
  2. Lack of Constructive Feedback: Focusing only on identifying problems without offering specific recommendations or solutions to address them is a missed opportunity to provide constructive feedback or suggestions for improvement. 
    • Bias, prejudice, or judgmental language: Use of neutral language is strongly advised. A reviewer may demonstrate bias or prejudice by favoring or opposing certain ideas, authors, or research groups without sound scientific reasoning. 
    • Unprofessional, rude, or offensive language: The use of unprofessional language, personal attacks, or offensive comments undermines the professionalism and collegiality expected in the peer review process. A review report should focus on the scientific content and should refrain from using derogatory or inflammatory language.
  3. Unethical peer review behaviors: 
    • Requests for self-citation: Suggesting citation of one's own published manuscripts when those manuscripts are irrelevant to the manuscript under review is an example of citation manipulation and can be considered scientific misconduct. JMIR Publications adheres to COPE guidance  on such matters (COPE: Citation Manipulation) and peer reviews may be removed from consideration in such situations.
  4. Lack of confidentiality: A poor-quality review report may breach the confidentiality of the peer review process by disclosing the reviewer's identity (What does the peer-review process at JMIR journals look like?), or sharing the content of the review report with unauthorized individuals. Peer reviewers are accountable for maintaining privacy and confidentiality of the peer review process.

Generative AI support in peer review

The full policy is published in this editorial on "Best Practices for Using AI Tools as an Author, Peer Reviewer, or Editor" in the Journal of Medical Internet Research. Peer reviewers are required to adhere to the policies with regards to peer reviewer accountability, transparency, and maintenance of confidentiality of any content that is generated by AI and incorporated into a peer review report submitted for any JMIR Publications journal.

Peer reviewers may also wish to carefully read the editorial policy to inform themselves of what JMIR Publications expects of authors when they use and disclose use of generative AI in producing parts of a manuscript submitted to a JMIR Publications journal. 

Because of the rapidly evolving nature of AI technologies, related policies, regulation, and best practices, JMIR Publications may periodically modify this editorial policy. It is the responsibility of the peer reviewers to adhere to these principles and practices provided.

**Peer reviewers are advised to carefully read the full published editorial policy on generative AI usage to ensure their adherence to the required appropriate statements and/or supplementary materials.**

Related Important Links

Footnote: This article was, in part, generated using ChatGPT by OpenAI, however, it was edited and enhanced by friendly humans from JMIR Publications staff.

Related articles