Ask a scientist—particularly at the start of their career—how journals work and you will either get a blank stare or a logical but inaccurate reply. The only people who know journal workflows are managing editors and chief editors. This chapter introduces the reader to the basics of journal function, what journals look for in a manuscript submission, and how they apply protocols of selectivity while maintaining fairness.
Journals are black boxes. A submitted manuscript goes in, and some time later a decision letter comes out. It is easy to imagine the basics of what goes on inside the black box. Someone does an initial screening; if the manuscript passes, then the paper goes to a specialized editor who does further screening, who, in turn may recommend desk rejection or be sufficiently impressed to send the paper out for external review. The reviewers eventually submit their reports, the editor makes the publication decision and the decision is emailed to the corresponding author.
Without any doubt, the most frequent question during my tenure as chief editor at Ecology Letters was “How do journals really work?”. My reply would typically be “Just as you would want journals to work!”. The curious colleague would then give their account of how they thought journals were run. Although most people got the basic inner workings right, they invariably missed the most salient feature:
Journals put everything at their disposal into identifying, improving and publishing choice science in the fairest way possible
Journals in a Nutshell
The modern-day journal derives from the four functions set out by Henry Oldenburg for the Philosophical Transactions in the seventeenth century and a fifth function enabled by the digital age[1]:
1. Registration. Dates (submission and publication) and author names are stamped on the article.
2. Certification. Quality control is ensured, usually through peer review.
3. Dissemination. The contents are intended for a specific scholarly audience, and readily accessible.
4. Archiving. A final version of each article is archived.
5. Discovery. The means by which the publisher enables scientists to find articles, text and data.
Journals achieve these through the use of a workflow applied and overseen by dedicated staff. Journals follow protocols at each point in the workflow, because they need to apply the same rules to each and every manuscript all the way from submission to the publication decision. Trained staff ensure every step of the workflow, and apply their expert judgment in decision making.
Journals live in challenging environments. Editorial offices need to cope with routine issues and unexpected problems and still obtain timely, thoughtful and thorough input from busy editors and non-recompensed reviewers. Doing this efficiently depends on staffing, organization, commitment of the editorial office and board and journal leadership.
Box 1. Why journals are secretive. There are two good reasons for why journals are secretive. First, they handle confidential information: authors’, reviewers’ and editors’ identities, comments and reports. Second, they base their reputations on service: the time it takes to render a publication decision, the quality/expertise of peer reviews and editorial assessment, and the fairness of the decision. Journals do not want what are sometimes complex situations to be misinterpreted by outside observers. Journals are also secretive for less laudable reasons. They can make questionable judgments and, knowingly or unknowingly, be biased (gender, geography, career stage. . .). Although journals may have internal checks and balances, they are rarely subject to independent inquiry.[2]
The Roles
Journals evaluate manuscripts and publish those that meet certain standards. Putting this into action requires experts in different roles.
Chief editor. The chief editor is responsible for journal decisions and in particular, publication decisions. The detailed responsibilities however vary from journal to journal. These can range anywhere from journal policy to following the progress of manuscripts from submission to decision, to only intervening in major events such as deciding whether to review a manuscript and making the publication decision. The chief editor is advised by members of the editorial board. Not all journals have a chief editor. Some are run by a managing editor or a committee of senior editors. Chief editors either have fixed tenures or renewable or open-ended terms. Some journals such as Nature and the Trends journals have professional chief editors, whereas most disciplinary journals are run by academic editors.
Editorial office. The editorial office is overseen by the managing editor and is the journal hub. It ensures smooth running of the workflow. The office notably liaises with the chief editor, the editorial board, reviewers, authors and the publisher. Editorial offices are increasingly managed by and within the publisher’s office.
Figure 1. Simplified diagram of journal operations, only showing interactions involving authors, the chief editor, the handling editor and reviewers. The editorial office oversees all operations, interacting with each of these four players to ensure smooth, efficient completion of each task.
Box 2. Workflows are how an editorial office organizes its handling of manuscripts. The workflow determines who actually does what and when. An editorial office will have a basic workflow that addresses all of the routine events from manuscript submission to the publication decision. The workflow will be flexible enough to deal with unanticipated issues. A well-run office will be consistent in the handling of each manuscript; that is, obtaining the evaluations and applying standards in making publication decisions in a timely fashion.
Editorial board. The board is a group of dedicated, reputable scientists who contribute their time and expertise to assessing the soundness of manuscripts. They represent the subject areas covered by the journal. Some editorial boards have section editors or associate editors who take on additional responsibilities. Depending on how the journal operates, the number of editors can vary anywhere from a few to more than 1000 (in the case of “megajournals”). Journals with fewer editors tend to have lower manuscript submission rates and require greater time investments from each editor.
Handling editor or “editor” is the member of the board to whom a manuscript is assigned. The editor will have qualifications (subject area, technical expertise) permitting her to evaluate the manuscript. Editors may be responsible for one or more of the following: recommending whether a submission goes for external review, choosing reviewers, acting as a reviewer, arbitrating reviewer reports, providing comments for authors and a publication recommendation to the chief editor, and advising on appeals. Their service term may be fixed or open.
Reviewers. Reviewers provide independent assessments of a manuscript. External peer review both extends the expertise of board members and reduces potential in-house bias. Reviewers are generally scientists at large, but some journals have an extended, dedicated editorial board that reviews manuscripts. Good reviewers (quality reports, reviewing frequently and on time) are sometimes invited to join editorial boards.
Authors are the source of a journal’s science. They usually choose the journal rather than the reverse, though some journals have dedicated editors who actively solicit author submissions. Journal choice is influenced by author needs (speed, likelihood of acceptance) and journal characteristics (reputation, impact factor, readership). Most journals do not differentiate the way a manuscript is handled based on whether authors are members of an academic society, editorial board members or scientists at large.
Readers are not only influenced by published content and transmit this through citations, but they are also potential future authors. “Reading” extends beyond literally reading articles, and includes text mining, extracting data and finding methods.
Academic societies. Some journals are overseen by an academic society that interfaces with the editorial office and editorial board, some managing all processes themselves (operating as a full publishing service) and others contracting with a dedicated publisher. Some academic societies have a role in determining editorial policies and the selection of editorial board members. Members of the society may have reduced costs when publishing in the society’s journal.
Publisher. The publisher is a company specializing in academic publishing. The publisher’s responsibilities include processing accepted manuscripts including assigning a DOI,[3] copyediting, proofreading, typesetting the manuscripts, managing the journal website and publicity at meetings and conferences, hosting pdfs and diverse metadata, printing and distributing paper copies of the journal and financing journal operations. Some publishers oversee journal operations including hiring the chief editor. Publishers are typically either a private commercial company, a university press or an academic society. The former is often referred to as “for-profit,” whereas the latter two are often called “not-for-profit”. All three types must have revenue that exceeds costs in order to survive and prosper, and while this is referred to as “profit” for companies, it is called “surplus” for university presses and societies.
Box 3. Gender bias. Helmer et al.[4] used public information about author, reviewer and editor identities from 142 Frontiers journals in the biological and physical sciences, engineering, health, the humanities and social sciences. When correcting for possible bias in their analysis, they found significantly lower than expected numbers of females as reviewers and authors and the same trend for female editors. There was considerable disparity among journals, with minima of about 10–15% women and maxima of 30–50% women in different roles. There are increasing trends in the representation of women over the period 2007 to 2015, and significant effects of same-gender preference, for example, female editors solicit more female reviewers than do male editors, and vice versa. One of the main issues not accounted for in this and many other studies is that women are still leaving science in large numbers—in part because of the hypercompetitive culture—despite moves to address imbalances. This makes the appearance of trends in gender equality seem overly optimistic.
What Authors and Journals Want
Journals and authors are both after impact. An impactful paper is perceived as interesting and important, and ultimately changes readers and science. Journals benefit from impact both in their image and through attracting more, potentially impactful submissions. Authors gain in their reputations and authority. In striving for impact, both journals and authors contribute to increasing knowledge and understanding.
Despite these similarities, there are subtle differences in what authors want in a journal experience, and what journals want from authors.
The author wants her manuscript to be published in the choice venue based on merit. “Choice” can mean many different things. An author is likely to try prestigious, high-ranking, reputable journals first, knowing that even getting peer reviewed would be an accomplishment. Should her paper be reviewed, she is prepared to conduct the revisions necessary. But, like many, she does not view revising as an enjoyable experience and so does what is required and reasonable to satisfy the opinions of reviewers and editors.
The journal wants to publish choice science. Each journal will have its own criteria of what “choice” science is, and they typically include one or more of: importance, novelty, topicality and scientific quality. Identifying “choice science” marshals the contributions of the many players in the previous section. Chief editors cannot possibly give the same amount of attention to all submissions, and therefore only spend significant time on papers that have a reasonably good chance of acceptance. To achieve this, journals put a premium on identifying the most likely papers to survive peer review. Those that do not make the cut are desk rejected.
Should the journal decide to have a paper peer reviewed, then they solicit external experts. Reviewer assessments will help the editor decide whether publication is still an option and, if so, the conditions necessary via a revision. Importantly—and in contrast to the authors’ own inclinations—the journal wants authors to amply satisfy reviewers and bring the study up to its fullest potential. The journal will encourage certain revisions and oblige others as a condition for further consideration or acceptance.[5] Journals are therefore in a position of power. The ideal experience for a journal is that reviewers identify each and every shortcoming and the authors address them all successfully and to the fullest extent.
Journals therefore search for more than what many authors are inclined to give. In the extreme, summary peer reviews that result in a manuscript being accepted with minor revisions may delight certain authors, but actually signal a failure in peer review. Like the many forms of reviewer bias, summary reviews do a disserve to authors, journals and to science.
Journal Selectivity
“Why did the journal reject my paper despite globally positive reviews?” Of course, there may be many reasons specific to a particular manuscript, but all else being equal, rejecting a good paper suggests that your study was competing with many other good papers and possibly some great papers. The journal was selective.
There are two basic models in the way journals select papers to publish.
The first are journals publishing all papers meeting acceptable scientific standards—so-called “soundness but not significance.”[6] Notable examples of such “megajournals” are PLoS ONE and Scientific Reports. These journals are of the philosophy that science deserves to be published regardless of its novelty or importance, or whether results are negative, positive or inconclusive, as long as the paper is in scope, the conclusions are supported by the data, and the article meets certain technical and ethical standards of reporting. Because such journals have fewer publication constraints, they tend to have high acceptance rates (typically greater than 50%), and because of these high rates, can apply economies of scale (since a higher percentage of submitted papers contribute to costs) and maintain lower publication costs compared with top ranking open access journals. [7]
The second—and vast majority—are journals that introduce added selectivity into their decision-making process. Selectivity may be in one or more of: scientific quality, novelty or importance. In making their decisions, journals will use some combination of a ranking system and the editor’s opinions. Other journals will add to this a limit in the number of articles published per issue, meaning that if submissions increase/decrease, rejection rates also change. Journals with increasing submission rates can increase their selectivity—and as a result, likely their impact factor as well—by simply keeping the number of accepted articles constant over time.
Box 4. Journals make mistakes. Just because a journal is selective and of high rank does not make it immune to error. These can range from publishing papers of low standard in statistical design and reporting to those that are fabricated or with fatal flaws and are subsequently retracted. [8]
Figure 2. Journal selectivity. Shaded areas beyond the vertical dashed lines show the manuscripts accepted for a given journal. (A) Journals that accept all manuscripts beyond a minimal standard. (B) Two journals with different levels of selectivity. The orange journal receives more higher standard (perceived importance, interest, scientific quality) submissions and can be more selective than the red journal.
The Challenge of Peer Review
Many of the above elements are unsurprising to scientists with some publication experience. You submit a manuscript to a journal, it either comes back desk rejected in days or perhaps a week, or goes out for review and you hear back from the editor, anywhere from 1–6 months later. If you have submitted many manuscripts over your career, then you know that occasionally something will go wrong, and you not only have had an unpleasant experience, but a good story as well. What you probably will not know are the underlying reasons.
Well-run journals are not immune from contentious publication decisions. One of the main culprits is not having enough expert information to make a decision that meets journal standards and author expectations. Usually, journals require two to three reviews, but some manuscripts—either because of their complexity or limitations in the scope or depth of reviewers—might require several or even more. There are however norms in peer review, and most editors are reluctant to go beyond four. Thus, a vigilant editor will solicit three reviewers with the desired expertise, and eventually get a fourth should she feel that there are major gaps. This highlights how options for additional reports are open early in the review process, but become few or disappear as time elapses and the decision approaches.[9] The handling editor is key in such situations, since it is she whom the chief editor can rely on to provide her own comments on the manuscript and, in cases of highly divergent reviews,[10] arbitrate in making her recommendation. It is not unusual that the editor’s views will diverge from those of one or more reviewers, and even contrast with the reviewers’ consensus. A frequent example is an editor who recommends major revisions, despite one or more reviewers suggesting rejection. In such instances the editor will provide explicit reasons to the chief editor and to the authors for disagreeing with reviewers.
Even the most professionally run journals disappoint. Obviously, no author enjoys having their manuscript rejected. And even situations where a manuscript is revised through one or more rounds and finally accepted can be perceived as a negative experience, particularly if the required revisions were arduous and were not (in the opinion of the authors) necessary.
To drive the above points home, below I present several procedural factors that can lead authors to believe that their manuscript was mishandled or the journal unfair in their decision.
Less Appropriate Reviewers
When a submission is to be sent for external review, the handling editor examines the manuscript and references therein, reviewer names suggested by the authors, her own reviewer choices and names suggested by the manuscript processing database. The editor then establishes an ordered list of reviewers based on a number of criteria, including no apparent conflicts of interest, appropriate scientific background, and quality and timeliness of the reviewer’s eventual previous assessments for the journal. The editor is therefore very much aware that some reviewers on the final list may be unknowns in terms of quality and reliability. She also cannot be sure whether previously flawless reviewers will be up to standard for the current review. Most importantly, the editor cannot oblige the preferred reviewers at the top of her list to accept the assignment. This means that some of the actual reports received may be from reviewers who are less experienced, somewhat outside of the manuscript’s main theme or less dependable in terms of time invested in the report.
Figure 3. Authors might imagine that the reviews were the first three invited (left), whereas in reality it can take many invites to get the required number of reports (right).
Late or Rushed Reports
Busyness is a problem for peer review, both because the star reviewers are often over-solicited and since those who accept to review can have unexpected demands on their time, meaning that their reports are delayed, never submitted or submitted but of low quality.
A late report can delay the publication decision, sometimes by months. A well-functioning editorial office will avoid soliciting reviewers who are habitually tardy and/or provide low-quality or summary reports. However, some reviewers who have good past records may be late with a report, and the editorial office will either wait until the report is submitted or, should multiple reminders go without a response, notify the reviewer that her report can no longer be received. In the latter case, the journal may solicit a replacement report, either from a member of the editorial board or an external reviewer who has expertise in the subject area and can be counted upon to provide a rapid assessment.
Box 5. Diversity promotes equity. Murray et al.[11] analyzed the outcomes of peer review from thousands of submissions to the biosciences journal eLife. Among their important findings were that women and authors from outside North America and Europe were both underrepresented as reviewers, and that mixed-gender and international teams led to more equitable (less homophilic) peer review outcomes.
Summary Reports
Summary reports (which are often also late) are equally challenging for an editor since, once submitted, they cannot be censured. Similar to reports from reviewers who are not experts in the area, an editor’s only recourse is to carefully arbitrate the report. Despite the best intentions of and work by the editor, a rejected manuscript with one or more lackluster reports can understandably result in authors putting the blame on the journal.
Opinions
Reviewers evidently have opinions and these rightfully enter into their evaluations. Opinions and suggestions can be useful for authors in their revisions and for editors in making publication decisions, but sometimes opinions are too extreme or demanding, and are given too much weight by editors. “I strongly believe . . . the authors should. . .” can be interpreted by authors as requiring what may be unnecessary revisions. “I suggest rejecting this manuscript” without a supportive basis can lead the unwary editor to recommending rejection to the chief editor. In contrast, “I strongly suggest accepting this paper. . .” appearing in the comments to authors will likely rile authors should their paper be rejected, even if the reasons provided for rejection are valid and clear.
Editors are responsible for delineating critiques of fact (flaws in experimental design, analysis, incorrect citations) from opinions and suggestions in reviewer reports. The former tends to have more weight both in the publication decision and necessary revisions. Sometimes editors will not comment on reviewers’ opinions and suggestions, and rather leave it to the authors to react and revise.
These points illustrate the challenges that editors face in ensuring decision standards and therefore fairness for authors. The question of who really has to be convinced for your manuscript to be accepted at a journal and how this may affect your writing strategy is addressed in the next chapter.
[1] Mabe, M.A., 2009. Scholarly publishing. European Review, 17(01), p.3 (and references therein). [2] There are some attempts to make journals more transparent, at least with regard to peer review, e.g. www.peere.org/peeer-in-a-nutshell/; http://www.bbk.ac.uk/news/birkbeck-to-investigate-the-peer-review-process-in-new-research-project [3] Digital Object Identifiers (DOIs) give a unique tag to an article, report or dataset. Before deciding on a journal to submit your work, verify that the publisher applies DOIs to their published articles. Most large publishers are members of Crossref, a registry organization for DOIs. [4] Helmer, M., Schottdorf, M., Neef, A., et al., 2017. Gender bias in scholarly peer review. eLife, 6, p.e21718. [5] Many disciplinary journals will facilitate the process by guiding authors in their revisions. [6] Wakeling, S., Spezi, V., Fry, J., et al., 2017. Open Access megajournals: The publisher perspective (Part 1: Motivations). Learned publishing: journal of the Association of Learned and Professional Society Publishers, 30(4), pp.301–311. [7] Björk, B.-C., 2015. Have the “mega-journals” reached the limits to growth? PeerJ, 3, p.e981; economies of scale also applies to large versus small publishers, which is why many society publishers partner with big publishers. [8] Fang, F.C. & Casadevall, A., 2011. Retracted science and the retraction index. Infection and Immunity 79(10), pp.3855–3859. Tressoldi, P.E., Giofré, D., Sella, F., et al., 2013. High impact = high statistical standards? Not necessarily so. PloS ONE, 8(2), p.e56180. Macleod, M.R., McLean, A.L., Kyriakopoulou, A., et al., 2015. Risk of bias in reports of in vivo research: A focus for improvement. PLoS Biology, 13(10), p.e1002273. Bohannon, J., 2013. Who’s afraid of peer review? Science, 342(6154), pp.60–65. [9] Under my tenure at Ecology Letters we always marshaled three reviews, and invited a fourth reviewer if there was any concern regarding limited reviewer expertise. [10] Assessor scores are at most weakly correlated, and weakly predict the eventual number of paper citations. Eyre-Walker A. & Stoletzki, N., 2013. The assessment of science: The relative merits of post-publication review, the impact factor, and the number of citations. PLoS Biology, 11(10), p.e1001675. [11] Murray, D., Siler, K., Larivière, V., et al., 2018. Gender and international diversity improves equity in peer review. BioRxiv DOI:10.1101/400515