AMA Manual of Style - Stacy L. Christiansen, Cheryl Iverson 2020
Editorial Assessment
Editorial Assessment and Processing
The principal goals of editing biomedical publications are to identify, select, improve, and disseminate scientific and clinical information that will advance the art and science of the discipline covered by the publication. For example, biomedical journals are a primary source for reporting research advances and for communicating information to improve medical care and public health. In addition to providing information to subscribers and other recipients of the journal, articles published in biomedical journals may be accessed and used by numerous other readers and constituencies, such as clinicians and researchers who seek information about particular topics; educators and opinion leaders who may use information from journal articles for teaching and informing colleagues; and members of the media, who frequently communicate information from research articles that involve medicine and health to the public.
However, perhaps the most important use of articles published in clinical biomedical journals is to provide valid and reliable information to physicians and other clinicians to promote the practice of evidence-based medicine, in which decisions about patient care are informed by acquiring, assessing, and applying relevant medical literature. These myriad uses of the biomedical literature illustrate the importance of journals having rigorous procedures for editorial assessment and processing to ensure the validity and improve the quality of published articles.
6.1 Editorial Assessment.
Editorial assessment of a manuscript ordinarily consists of 3 phases: initial editorial review, peer review, and editorial assessment and decision-making (Figure 6.1-1). During the initial editorial review, editors assess submitted manuscripts for overall quality and appropriateness for the readership of the journal. Manuscripts that do not pass this initial editorial review are rejected, whereas those that pass this initial evaluation proceed to the peer review phase. Peer review (see 6.1.2, Peer Review) involves evaluation of the manuscript by reviewers who have expertise in the topic and knowledge about the information reported in the manuscript and may include evaluation by reviewers with expertise in specific aspects of the manuscript, such as statistical reviewers. After completion of peer review, editors reassess the manuscript with consideration of the reviewers’ comments and suggestions as part of editorial decision making. The integrity of the editorial assessment process requires strict confidentiality on the part of editors, journal staff, and peer reviewers and careful attention to possible biases and conflicts of interest of authors (see 5.7.1, Confidentiality During Editorial Evaluation and Peer Review and After Publication).
Figure 6.1-1. Example of Editorial Assessment
6.1.1 Manuscript Assessment Criteria.
Several criteria are central to the evaluation of manuscripts submitted for publication, including importance, validity, and quality. Assessment of importance involves determining whether the manuscript reports information that represents a scientific advance (recognizing that individual articles usually convey only small advances), has clinical relevance (if the journal is to be read and the information used by practicing clinicians), is sufficiently novel to add new scientific information to the field, and will likely be of interest to readers. An additional component of importance is editorial priority, a composite judgment made by the editor regarding the merits of a particular submission relative to the merits of other submissions under evaluation at the same time, weighed in the context of articles the journal has recently published, has scheduled for publication, or has under consideration.
Evaluation of validity of a research report involves critical assessment of the internal integrity of the manuscript to determine whether the research design and study execution support the findings, inferences, and interpretations reported in the manuscript. For original research reports, assessment of validity involves consideration of whether the design and methods are appropriate to answer the stated research questions; the research questions and the methods used to answer them are well described and rigorously conducted; the data analysis is appropriately performed; the conclusions are supported by the results; and patients or research participants were treated ethically (see 5.8, Protecting Research Participants’ and Patients’ Rights in Scientific Publication).
Assessment of quality of a research manuscript involves determining how well the authors present the information, such as overall organization and cohesiveness of the manuscript, clarity of the purpose of the study and description of the methods, logical and transparent reporting of data, including effective use of well-constructed tables and figures, appropriate and objective interpretation of the study findings, and inclusion of key and suitable references. Other aspects of quality assessment involve evaluating the quality of the writing and logical presentation of information and how well the authors follow the journal’s guidelines or instructions for authors.
For research and review manuscripts (see 1.0, Types of Articles), writing quality and clarity may influence the evaluation by editors and peer reviewers even though the importance and quality of the research should be the main focus for assessment. Writing quality can be improved by manuscript editing (see 6.2.1, Manuscript Editing) but only if the research is described with sufficient clarity to permit basic understanding. Writing quality and presentation of information may be particularly important factors in the assessment of opinion articles, such as Viewpoints or invited opinion pieces, in which well-organized, well-argued, and well-written compelling manuscripts generally are more likely to receive higher priority for publication than poorly written manuscripts.
For research reports, the specific nature or direction of results should not ordinarily be a major factor in quality assessment. If a rigorously conducted investigation addresses an important clinical or scientific question, uses high-quality and rigorous methods, and has findings that are determined to be valid, the study results may be worth publishing regardless if “positive” (demonstrates an effect or an association) or “neutral” or “negative” (does not demonstrate an effect or association). Publication bias could result from a tendency for investigators not to submit or for editors not to accept manuscripts that do not report statistically significant positive results. However, depending on the topic being investigated, a well-conducted, adequately powered study that shows that a particular intervention is not effective may be as important as a study that reports a positive result.
For review articles and other reports that do not involve original research, editorial assessment involves determining the importance and relevance of the topic for the journal readership; the completeness, accuracy, timeliness, logical presentation, and, usually, clinical utility of the information presented; and the quality and rigor of the evidence provided to support recommendations, for example, for or against diagnostic testing or treatment.
6.1.2 Peer Review.
Peer review was first used for biomedical publications by the Royal Societies of London and Edinburgh in the 18th century but evolved haphazardly, was not used consistently until after World War II,1,2 and has only come under scientific scrutiny since the 1980s.3,4 Despite the routine use of peer review of manuscripts by scientific journals, the peer review process has been criticized for its reliance on human judgments that may be subject to biases and conflicts of interest. Moreover, until relatively recently, there was little empirical evaluation and documentation of the efficacy and value of the peer review process. However, with the advent of the International Congresses on Peer Review and Biomedical Publication in 19895 and subsequent congresses held every 4 years since then,6 empirical research on editorial peer review and scientific publication has examined many aspects of these important practices, providing evidence-based information to better understand and improve the peer review process, the quality of research reporting, and the scientific publication process.
The essence of peer review for biomedical journals consists of asking experts, “How important and how good is this manuscript, and how can it be improved?” The involvement of expert consultants to advise editors about importance, validity, quality, and improvement of manuscripts has become a standard quality assessment measure in biomedical publication. Review of a manuscript by experts in the subject matter being reported is helpful to assess importance and quality of the work and to determine the context for the information being reported, such as whether study findings represent novel advances or are incremental additions to the scientific literature.
Peer reviewers should assess all components of a manuscript, including the title, abstract, and text of the manuscript, the references, the tables and figures, and all supplementary material. On the basis of this assessment, reviewers should provide an overall assessment of the manuscript, along with comments for the authors regarding the strengths and weaknesses of a manuscript, including specific methodologic or substantive issues that need to be addressed, as well as suggestions for improvement. Although specific criticisms and suggestions are much more valuable than summary judgments, reviewers also may be asked to provide an overall recommendation to the editor about suitability of the manuscript for publication in the journal. Although these suggestions are helpful, the comments and recommendations about publication from the peer reviewer consultants are considered advisory, and all decisions about rejection, revision, or acceptance are the responsibility of the journal editors (see 5.11.4, Editorial Responsibility for Peer Review).
6.1.2.1 Selection of Reviewers.
The selection of peer reviewers and the number of reviewers for evaluating a particular manuscript are matters of editorial judgment. In most cases, peer reviewers are usually experts who are not part of the journal staff or otherwise not associated with the journal, although members of the editorial staff or editorial board of the journal may serve as peer reviewers in areas of their expertise. The editor’s knowledge of experts in a particular field often determines reviewer selection. Many journals maintain a database of reviewers indexed by areas of expertise, including information on review quality and turnaround time. The reference list of a manuscript may be a useful starting point for identifying potential peer reviewers who have been contributors to the literature on the same topic. A literature search by the editor can also be helpful in identifying potential reviewers.
Authors may suggest names of possible peer reviewers, and some journals encourage authors to make these suggestions, although some reviewers recommended by authors may provide more favorable reviews than reviewers selected by editors without suggestions from authors. At times, authors also may request that some persons not serve as reviewers for their manuscript, usually because of perceived bias. Editors should consider such requests and information, but the selection of peer reviewers is the responsibility of the editor, who must use judgment not only in selecting the reviewers but also in distinguishing a reviewer’s valid praise or criticism from unwarranted bias for or against a particular manuscript. Reviewers are expected to disclose to the editor any conflicts of interest they may have regarding a topic or an author at the time they are invited to review the manuscript (see 5.5.6, Requirements for Peer Reviewers, and 5.11.4, Editorial Responsibility for Peer Review).
Evaluation of the scientific validity of original research reports usually requires peer review by reviewers with expertise in statistics, epidemiology, and research methodology to provide assessment of study design and research methods. For some journals, some statistical reviewers are members of the editorial staff or may serve as paid consultants. Epidemiologic and statistical review can be helpful in identifying weaknesses in study design and methods and in improving scientific reports for publication. Moreover, with the use of increasingly sophisticated research and statistical methods, having input from peer reviewers with specific methodologic expertise has become increasingly important in the assessment of the validity and quality of scientific reports.
6.1.2.2 Concealing of Author and Reviewer Identities.
Scientific journals usually adopt one of several models for conducting peer review that involve revealing or concealing the identities of authors and peer reviewers. Journal policies vary regarding whether author identities, reviewer identities, both, or neither are kept confidential or are revealed, and these practices should be indicated in the journal’s information for authors (see 5.11.4, Editorial Responsibility for Peer Review).
Biomedical journals commonly use a single-blind (also called single-masked) review process in which authors’ identities are revealed to reviewers, but the names of reviewers are not revealed to authors (see 5.7.1.1, Confidentiality Requirements During Blinded [Anonymous] Peer Review). This process recognizes the difficulty of concealing author identities, makes it easier for reviewers to detect certain forms of research irregularities (such as attempts at duplicate publication by the same authors), and may encourage more candid reviews because the reviewers know they are anonymous to the authors, who may be their professional colleagues. However, this single-blind approach may have some potential disadvantages. For instance, some reviewers might be influenced by knowing the identities and reputations of authors or their affiliations and therefore might not judge a manuscript solely on quality and importance.
Another approach involves a double-blind (also known as double-masked) peer review, in which neither the authors’ nor the reviewers’ identities are revealed. Authors who submit a manuscript to a journal that attempts to conceal author identities may be instructed to remove identifying information from all parts of the manuscript and to submit that information separately, such as author names, affiliations, and acknowledgments (including funding sources). Theoretically, the double-blind approach should contribute to a more objective evaluation of the manuscript because reviewers may be more likely to evaluate the manuscript based on quality, without being influenced by knowledge of the identities or affiliations of the authors. However, concealing author identities is not always successful because of self-referencing in the manuscript or reviewer knowledge of the authors’ work. There is limited evidence to support the idea that reviewers who are unaware of authors’ identities will provide more objective reviews (see 5.7.1.2, Confidentiality Requirements During Double-blind Peer Review).
A third approach used by some journals involves open (or open-identity) peer review, in which reviewers are aware of the identities of the authors, and authors are aware of the identities of those who reviewed their manuscript. Advocates of this approach to the manuscript review process support the importance of transparency in science and suggest that authors should know who is evaluating their work and that reviewers should stand by their critiques by signing them. In one study,7 asking reviewers to consent to being identified to authors had no important effects on quality of reviews, recommendations regarding publication, or the time taken to review but increased the likelihood of reviewers declining to review (see 5.7.1.4, Open Review).
Regardless of the approach used for peer review, and despite its limitations, peer review has generally been considered “indispensable for the progress of biomedical science.”8 So far, convincing evidence has not demonstrated that a better alternative has emerged for the assessment and improvement of manuscripts submitted to biomedical and scientific journals.
6.1.3 Editorial Decisions.
On the basis of the evaluation of the manuscript by the editors and, for manuscripts that have undergone peer review, consideration of the comments of the peer reviewers, submitted manuscripts are rejected, returned to authors with an invitation to submit a revised manuscript, or, on rare occasions, accepted without revision. For some journals, a substantial proportion of manuscripts are rejected on the basis of the initial editorial review and assessment, a smaller proportion are rejected after peer review, and an even smaller proportion proceed to revision and reevaluation.9
Although journals may have different approaches and models for editorial decision-making, the editor in chief has ultimate responsibility for all editorial decisions for rejection, revision, and acceptance of submitted manuscripts. For some journals, the editor in chief may delegate the responsibility to deputy editors or associate editors to make decisions about rejecting manuscripts after initial review and assessment, perhaps with a second opinion about the decision from another editor. Many journals may hold meetings (in person or by teleconference, videoconference, or internet communication) during which submitted manuscripts and their reviews, and also revised manuscripts, are presented and discussed among the editors before decisions are reached regarding proceeding with revision or acceptance for publication.
For manuscripts that are rejected without external peer review, editors may choose to provide brief comments that indicate the reason for the decision. For manuscripts that are rejected after peer review, the editors usually will provide the authors with comments from the peer reviewers and perhaps comments from the editors, so authors can consider this information as they revise the manuscript for submission to another journal.
Reports of original research and other major articles almost always undergo peer review, statistical review, and revision before acceptance for publication (see 1.0, Types of Articles), whereas manuscripts that express opinions, such as Viewpoints or Editorials, may be accepted after revision based on editorial review and evaluation.
6.1.4 Revisions.
If the editors decide to request a revision of a submitted manuscript, the authors should receive detailed recommendations from the editor about what is expected in the revision, instructions about how to improve the manuscript, and detailed comments from the peer reviewers. Guidance from the editor is particularly important if recommendations from the peer reviewers are discordant. In addition, at some journals, the editorial staff may conduct internal review of the manuscript and provide additional detailed comments regarding manuscript format, methods, data analysis, interpretation, presentation, and other issues for improving the manuscript. Authors are usually requested to submit a detailed response that addresses the comments, indicates how the revisions were completed, and provides reasons for any suggested revisions not undertaken when they return the revised manuscript. In most cases, it is advantageous for the authors to revise the manuscript promptly and thoroughly and to submit the revised manuscript within a relatively short time, perhaps 2 or 3 weeks.
However, authors should realize that a request for revision does not guarantee that the revised manuscript will be accepted for publication. Revised manuscripts are subject to editorial review and reevaluation and perhaps additional peer review. Important issues with the study methods or manuscript content may not become apparent until after revision. For some manuscripts, several rounds of review and revision may occur before a final decision is reached (see 5.11.6, Editorial Responsibility for Revision).
After submission of a revised manuscript that satisfactorily addresses the comments of the editors and reviewers, the editors will reevaluate the manuscript and make a decision about publication. Manuscripts often are accepted provisionally, contingent on authors fulfilling additional requested revisions or providing required items, such as completed authorship forms and copyright transfer statements.
6.1.5 Appealing an Editorial Decision.
If a manuscript is rejected, authors occasionally will appeal the decision and request reconsideration, most often because they perceive that the reviewers or the editor may have misjudged the importance of the submission. Journals should develop procedures for responding to these requests for appeal of editorial decisions. The editor in chief should be involved in the evaluation of the authors’ request for reconsideration and the reasons for the request, with careful reassessment of the initial manuscript and the comments of the peer reviewers and detailed discussion with the editor who rendered the initial decision. In most cases, the initial editorial decision is upheld, unless the authors can provide objective and compelling grounds for reconsideration of the original decision, particularly if they can provide new data or new analyses, as opposed to differences of opinion about editorial priority (see 5.11.5, Editorial Responsibility for Rejection).
6.1.6 Postpublication Review.
Postpublication review in peer-reviewed journals may include several potential mechanisms: letters to the editor about the published article that identify flaws, raise additional substantive concerns, or discuss additional important implications of the findings; online responses to published articles; efforts to replicate the work; and the reactions from clinicians applying the information in practice. Such evaluations are important for ensuring responsible scientific dialogue about published articles. Journals should encourage submission of letters to the editor raising issues and questions about published articles, and authors of the published article should be encouraged to prepare responses to those critiques. Publication of these scientific exchanges in the Letters section provides an avenue for postpublication review and discussion (see 5.11.8, Correspondence [Letters to the Editor]). Editors should also perform a quality review of each published issue of their journal to identify areas of content and format that can be corrected or improved in subsequent issues (see 5.11.14, Editorial Quality Review).
6.1.7 Corrections.
As part of postpublication review, errors may be identified by authors or readers, editors, or other sources. Correction of errors is important to ensure the accuracy of the scientific record.10 In most cases, corrections are minor and straightforward. The Correction notice is listed in the Table of Contents (in print, online, or both), and the Correction should be published in a specific section. For example, the JAMA Network journals publish Corrections at the end of the Letters section. Corrections should be indexed, with a reference and online link to the original article, thereby enabling online database services (such as MEDLINE) to link indexed articles with published corrections (see 5.11.8, Correspondence [Letters to the Editor], and 5.11.10, Corrections [Errata].) The erroneous information should be corrected in the online versions (HTML and PDF) of the published article, with the reason and date of the Correction prominently displayed. Corrections should also be linked from the article to the Correction on the journal’s website and appended to the article PDF. If online-only corrections are made or if corrections are made online before the article appeared in print, the changes and date of the correction should be indicated in the electronic file. In some cases of pervasive error in a published article, such as when an error was inadvertent (eg, as may occur with incorrect coding of data), and the error resulted in incorrect data points throughout an article, yet the underlying science is still reliable and important, editors may consider retracting the article that contains the errors and publishing a corrected replacement article, with explanation from the authors.11
For journals that publish articles online first (eg, before republication in a print/online issue), corrections may be incorporated before the second publication in the print/online issue. The erroneous information should be corrected in all online versions (HTML and PDF) with the reason for and date of the correction prominently displayed.
For example, the JAMA Network journals include the following on page 1 of corrected articles in the PDF version: “Corrected on September 20, 2018.”
In addition, in the Article Information at the end of the article, the following may be included:
“Correction: This article was corrected for errors in Table 1 on May 29, 2017.”
Similarly prominent notes on the HTML version of the article and Correction notice should be displayed with reciprocal links. For example, the JAMA Network journals include the following at the top of a corrected article and Correction notice:
On the corrected article: “This article was corrected | View correction”
On the correction notice: “A correction has been published | View article”