A recent report by the International Association of Scientific, Technical and Medical publishers has estimated that there were about 33,100 active scholarly peer-reviewed English-language journals in mid-2018, which collectively published about 3 million articles a year. The number of scientific journals had increased steadily by 3.5% per year but this has recently jumped to 5% in order to meet the requirements of an equally growing scientific community, which is estimated to have around 7–8 million members globally. There is no doubt that this does significantly impact on the number of average review requests a researcher receives, and in total it is estimated that 13.7 million reviews took place in 2016. Peer review remains the most widely used form of scientific validation prior to publication, and so it is inevitable that the increasing number of journals, and submissions, is having a significant impact on the process and more generally the scientific community.

We are sometimes asked how editors, when faced with an increasing number of papers, are able to find the ‘right’ reviewers. What are the criteria editors follow when selecting candidates and how do they ensure and maintain the quality of the reports they receive?

Motivated by Peer Review Week and this year's theme of ‘Quality in Peer Review’, we addressed, in our previous editorial, the practice of ghost-writing reviewer reports1. Here, we continue with the theme of quality and focus on the procedures we follow when initiating the peer-review process. We will explain how we find and match reviewers to a particular paper and some of the factors we consider when assessing their reports. Despite increasing demand, as a journal, we strive to maintain a high standard of peer review and ensure that every paper sent out for assessment is fairly and carefully evaluated by a balanced and capable group of reviewers.

All papers received by Communications Physics are first assessed by one of our editors. Editors at Communications Physics are either in-house professional editors or editorial board members (EBMs). The major difference between the two is that EBMs are still active researchers with specialised technical expertise as well as an up-to-date perspective on their particular research area (for more information on the role of EBMs see2 a recent editorial by our sister journal Communications Biology). All editors are trained to establish the strengths and weaknesses of a study and how these align with the aims and scope of the journal. They identify the main claims of a paper, the supporting techniques and data and the specialist knowledge required to assess these claims. For papers deemed suitable for formal review, an editor then matches referees to those skill sets. This is most successfully achieved by delving into the literature to identify scientists and institutes that have published related work. Editors can develop an extensive knowledge of researchers in the field through published papers, past reviewers or from attending conferences and seminars.

When approaching someone for a review there are multiple factors taken into account. Papers present typically a combination of different areas of expertise and methods, and to be assessed correctly requires experts with that specific knowledge. Some of these will be more integral to the study and require greater scrutiny than others.  The fundamental questions to consider are whether the study is experimental, theoretical or a combination of both. Do the results disagree or agree with previous studies? If so should those involved be given the opportunity to provide an opinion, and if these individuals are approached, would this pose as a conflict of interest?

Contrary to popular belief we do not only seek top researchers in the field. We also believe that reports are the most informative when they come from a diverse range of individuals and we try to seek diversity in gender and geographical location as well as career stage. Above all, Communications Physics is interested in those at the front line of research and we believe that this is particularly important for a specialised journal. For instance, early career researchers can have intricate knowledge of a particular subject and this is often reflected by detailed and focused reports. On the other hand, those with more experience benefit from a more expansive understanding of the field and are skilled at putting research into its proper context with the current literature.

Regardless of experience, we always require that a reviewer has demonstrable understanding of what needs to be assessed, as determined through their publication history or recorded reviewer activity from external services like Publons, their website and/or previous interactions. At times, reviewers who are unable to report recommend an alternative referee, a collaborator, or a more junior colleague. We very much welcome these recommendations. Most importantly, Communications Physics hopes to maintain and develop relationships with the research community. Those who have either published or reviewed for us have, by experience, a better understanding of our aims and scope, and we hope that they would be motivated to contribute to the journal on more than one occasion. As a matter of fact, authors and referees belong to the same community, and provide their contribution from two different sides each time, sometimes having opposite expectations depending on whether they are authors or referees. For example, the same author who expects a decision within less than a month, may take a whole month to provide a referee report.

There are of course circumstances where referees are not contacted, regardless of their scientific suitability to review a specific study. This is most typically because of conflict of interest, for instance when a referee is based at the same institution as the authors or they have previously published work together. If authors wish to exclude a reviewer this is honoured, unless the authors have excluded an unreasonable number of candidates: the more specialised a study the more limited becomes the number of experts available. A large number of reviewer exclusions will ultimately result in a delay in reaching a decision.

When assessing a report, editors do not simply count the number of yeses and noes. The value of a report is in its detail and clarity; criticism as well as praise are more convincing if they are supported with appropriate evidence. Each reviewer has their own voice and brings a different perspective to a study. The editor interprets reports in order to highlight the most relevant aspects. This process is nuanced and complex and does not always have a straightforward answer. Some reviewers express their opinion more subtly than others and misinterpretation can occur. Editors do not blindly follow referees’ advice, and their decision takes into account the editor’s original evaluation as well as the reviewer reports. Editors also understand that a reviewer’s assessment may not always be perfect and could contain technical inaccuracies or professional bias. We ask reviewers to declare any such conflicts of interest before accepting to review a paper and this is something we ourselves always check for at every stage of the review process. Authors (and other reviewers) are also encouraged to highlight such instances as early as possible so that an editor can take this into account when making a decision.

The value of a report is in its detail and clarity; criticism as well as praise are more convincing if they are supported with appropriate evidence.

While the final decision on every paper lies with the editor, we would not be able to reach a decision without the wealth of technical and more general insight that reviewers can provide. Therefore, finding and communicating with the best reviewers for a paper is an integral skill for an editor to develop and one we are always seeking to improve. Ultimately, assessing a paper is a collaborative effort between reviewers and editors and is a relationship that at Communications Physics we wish to continue to strengthen as we grow as a journal.