The Lost Art Of Reviewing: Hints for Reviewers and Authors

(source via Geographical Perspectives)

The last three months have been largely dedicated to the review of publications: On the one hand, some of my own work was going through the review-process, while on the other hand, I myself had to review several publications for various journals. During this time I got to see the review reports of fellow reviewers, both for my own work and the work I had to reviewed. Because the peer-review experience is an integral part of modern science, some hints for both authors and reviewers:

For reviewers:

• Do you have time?

  When you get your first request to review a paper for a peer reviewed journal^‡, this is an exciting experience. It implies you are being recognized by the community as a scientist with some merit. However, as time goes by, you will see the number of these requests increase, and your available time decreases (this is a law of nature). As such, don’t  be too eager to press that accept button. If you do not have time to do it this week, chances are slim you will have time next week or the week after that. Only accept when you have time to do it NOW. This ensures that you can provide a qualitative report on the paper under review (cf. point below) No-one will be angry if you say no once in a while. Some journals also ask if you can suggest alternate reviewers in such a case. As a group group leader (or more senior scientist) this is a good opportunity to introduce more junior scientists into the review process.

  ^‡ Not to be mistaken with predatory journals, presenting all kinds of schemes in which you pay heavily for your publication to get published.

• Are you familiar with the subject?(material/topic, theoretical method, experimental technique,…)

  You have always been selected specifically for your qualities, which in some cases means your name came up in a google search combining relevant keywords (not only authors and reviewers are victims of the current publish-or-perish mentality). Don’t be afraid to decline if a paper is outside your scope of interest/understanding. In my own case, I quite often get the request to review experimental papers which I will generally decline, unless I the abstract catches my interest. In such a case, it is best to let the editor know via a private note that although you provide a detailed report, you expect there to be an actual specialist (in my case an experimentalist) present with the other reviewers which can judge the specialized experimental aspects of the work you are reviewing.

• Review a paper without checking out the authors.

  In some fields it is normal for the review process to be double blind (authors do not know the reviewers, and the reviewers do not know the authors), in others this is not the case. However, to be able to review a paper on it’s merit try to ignore who the authors are, it should reduce bias (both favorable or unfavorable), because that is the idea of science and writing papers: it should be about the work/science not the people who did the science.

• Provide a useful review (positive or negative)

  Single sentence reviews stating how good/bad a paper is, only shows you barely looked at it (this may be due to time constraints or being outside your scope of expertise: cf. above). Although it may be nice for the authors to hear you found their work great and it should be published immediately, it leaves a bit of a hollow sense. In case of a rejection, on the other hand it will frustrate the authors since they do not learn anything from this report. So how can they ever improve it?

• Nobody is perfect, and neither are our papers.

  No matter how good paper, one can always make remarks. Going from typographical/grammatical issues (remember, most authors are not native English speakers) to conceptual issues, aspects which may be unclear. Never be afraid to add these to your report.

For authors:

• Do not submit a draft version of your paper.

  Although this is a quite a obvious statement, there appear to be authors who just send in their draft to a high ranking journal to get a review-report and then use this to clean up the draft and send it elsewhere. When you submit a paper you should always have the intention of having it accepted and published, and not just use the review progress to point out the holes in your current work in progress.

• Take time to create Figures and Tables.

  Some people like to make figures and tables, others don’t. If you are one of the latter, whatever you do, avoid making sloppy figures or tables (e.g. incomplete captions, missing or meaningless legends, label your axis, remove artifacts from your graphics software, or even better switch to other graphics software). Tables and figures are capitalized because they are a neat and easy to use means of transferring information from the author to the reader. In the end it is often better not to have a figure/table than to have a bad one.

• We are homo sapiens pan narrans

  Although as a species we are called homo sapiens (wise man) in essence we are rather pan narrans (storytelling chimpanzee). We tell stories, and have always told stories, to transfer knowledge from one generation to another. Fairy-tales learn us a dark forest is a dangerous place while proverbs express a truth based on common sens and practical experience.

  As such, a good publication is also more than just a cold enumeration of a set of data. You can order you data to have a story-line. This can be significantly different from the order in which you did your work. You could just imagine how you could have obtained the same results in a perfect world (with 20/20 hindsight that is a lot easier) and present steps which have a logical order in that (imaginary) perfect world. This will make it much easier for your reader to get through the entire paper. Note that it is often easier to have a story-line for a short piece of work than a very long paper. However, in the latter case the story-line is even more important, since it will make it easier for your reader to recollect specific aspects of your work, and easily track them down again without the need to go through the entire paper again.

• Supporting/Supplementary Information (SI) is not a place to hide your work

  Some journals allow authors to provide  SI to their work. This should be data, to my opinion, without which the paper also can be published. Here you can put figures/tables which present data from the publication in a different format/relation. You can also place similar data in the SI: E.g. you have a dozen samples, and you show a spectrum of one sample as a prototype in the paper, while the spectra of the other samples are placed in SI. What you should not do is put part of the work in the SI to save space in the paper. Also, something I have seen happen is so-called combined experimental-theoretical papers, where the theoretical part is 95% located in the SI, only the conclusions of the theoretical part are put in the paper itself. Neither should you do the reverse. In the end you should ask yourself the question: would this paper be published/publishable under the same standards without the information placed in SI. If the answer is yes, then you have placed the right information in the SI.

• Sales: 2 papers for the price of one

  Since many, if not all, funding organisations and promotion committees use the number of publications as a first measure of merit of a scientist, this leads to a very unhealthy idea that more publications means better science. Where the big names of 50 years ago could actually manage to have their first publication as a second year post doc, current day researchers (in science and engineering) will generally not even get their PhD without at least a handful publications. The economic notion of ever increasing profits (which is a great idea, as we know since the economic crisis of 2008) unfortunately also transpires in science, where the number of publications is the measure of profit. This sometimes drives scientists to consider publishing “Least Publishable Units”. Although it is true that it is easier to have a story-line for a short piece of work, you also loose the bigger picture. If you consider splitting your work in separate pieces, consider carefully why you do this. Should you do this? Fear that a paper will be too long is a poor excuse, since you can structure your work. Is there actually anything to gain scientifically from this, except one additional publication? Funding agencies claim to want only excellent work; so remind them that excellent work is not measured in simple accounting numbers.

Disclaimer: These hints reflect my personal opinion/preferences and as such might differ from the opinion/preference of your supervisor/colleagues/…, but I hope they can provide you an initial guide in your own relation to the peer-review process.