nextgenforensic

Tips for peer reviewing scientific articles

Caoilte Ó Ciardha & Kelly M. Babchishin

Over the last year or so, we’ve started new roles as associate editors at the Journal of Sexual Aggression (CÓC) and Sexual Abuse (KB). Transitioning from simply presenting opinions for consideration to making the decisions on people’s work has been daunting but eye-opening. For example, you would not believe the amount of people who turn down reviewing. We get it, reviewing is a hassle, and when that one-week1 reminder arrives telling you the review is due you will invariably curse your past self who naively assumed you would have somehow cleared the steaming pile of work off your desk to make room for it.

Reviewing is also a difficult art, especially when you are starting off on your academic career. Below are some suggestions that may help. These suggestions are not to make your review more helpful to the editor (though it is a great secondary outcome we would appreciate). Instead, they are meant to improve your review to better highlight the main issues of the manuscript and, when appropriate, to increase the helpfulness and clarity of your comments so that the authors can more easily address your comments and provide an improved second submission.  It’s not an exhaustive list and you may disagree with some. Feel free to add your own thoughts in the comments section.

 

Accept review invitations

Think about the last manuscript you submitted. How many journals did you send it to? How many rounds of reviews did it face before finally getting accepted (hopefully, it got accepted)? Chances are between 2 and 6 people agreed to review your manuscript and some of them probably reviewed it multiple times. These reviews are essential in ensuring the integrity and quality of peer review publications. Therefore, you should be agreeing to review a minimum of between 2 and 6 papers for every one you submit. Let’s say 4. You could divide this number by your average number of co-authors, if you tend to publish with people who are also available to review. Our advice, however, is that saying “yes” should be your default. For those that have a paper under review most months of the year, this means you should always have two or three manuscripts to review in your ‘to do’ list. It is just good karma, and good for your field. It is also a good way to get keep abreast of the development in your field of expertise.

Don’t just review the ones that you know will be good based on the abstract

We’ve often heard colleagues say that they cherry-pick papers based on how ‘good’ the abstract reads. Weaker papers need good reviewers too. What happens to the literature if the only people to say ‘yes’ to reviewing less stellar papers are the ones who don’t realise how problematic they are? Instead of the perceived ease of the review, your decision to review a particular manuscript should be based on your expertise.

Structure your review

Everyone has their own style of reviewing. Mine (CÓC) is heavily influenced by the first reviews I ever received. I usually start with some positive things about the paper but will also point out in broad terms my main concerns. I then have a section of general comments followed by more specific comments. As much as possible, I try and give a location in the manuscript, especially for specific comments, using page and line numbers, if the manuscript has them (page and paragraph if not). Some reviewers (KB) structure their reviews by summarizing the main suggestions firsts and then taking each section of the manuscript in turn. Whichever approach you take, the best reviews clearly (and concisely) highlight the main issues.

Focus on the key things

Especially when you’re reviewing the first submission of a manuscript, the editor wants you to help them answer the following questions:

  • Is it clear what the purpose of the study is?
  • Does the reader get the background literature to understand the research question?
  • Is the literature review well organized and complete?
  • Is the research question justified?
  • Is the methodology justified? Does it answer the research question(s)?
  • Is there enough information for replication?
  • Are the stats appropriate and do they appear correct?
  • Are the results clear and properly interpreted?
  • Are implications and limitations sufficiently examined?
  • Are the main study findings integrated into the existing literature? At the end of the day, a study is part of an academic conversation and, therefore, should address prior relevant conversations and, ideally, also provide a revised, convincing conclusion.
  • Are there any issues with the manuscript in terms of writing style, conciseness, and overall narrative.

 

Don’t assume the action editor will agree with you

It is best not to recommend acceptance or rejection in the comments to the authors as the editor may disagree with you. Instead, you can use some hedging to avoid committing yourself to a decision: “if submitting a revised manuscript, the authors should…”. Use the reviewing portal functions to make your suggestion to the editor.

Offer solutions, when possible

Sometimes your concerns cannot be addressed other than the authors noting the concern (e.g., limitations of a particular method) or refocusing the study purpose to focus on the stronger aspects of a particular study. Other times, however, your concerns can be addressed. In those cases, do not just summarize your concern; provide a possible solution or direction to the authors. Engage with authors. Provide guidance. If the introduction is too brief, note a few themes or references that could bolster it. If the analyses are not appropriate/limited, provide a reference (or references) to possible alternatives. If the findings are not well structured or clearly explained, offer some suggested methods of structuring and explaining the findings. If the conclusion is lackluster, offer a few suggested themes that could be addressed. As authors, we know that the most useful reviewers are those that offer suggestions to their concerns rather than simply listing the issues with the manuscript. Return the favour by giving suggested solutions in your reviews. The authors may not agree to all of them, but at least you’ve provided them with a starting point and some guidance.

If the manuscript needs a major rewrite, there’s no need for an exhaustive list of typos or other minor issues

It would be much more useful to give general constructive feedback than hammering home the use of the Oxford Comma, or the presentation of the findings, if you believe a new type of analyses should be used or a major rewrite is required. The more fine-grained suggestions could come on revision 2 or 3. In fact, typos shouldn’t be part of your job as a reviewer. If the manuscript is riddled with errors, highlight a sample and move on. Often, if the manuscript requires considerable revisions (think complete rewrite or complete reanalysis), the reviewers’ comments are limited to 2-3 key issues. If the authors resubmit, the second set of reviews becomes much longer and include more minor issues, such as the presentation of results or the comprehensiveness of the discussion.  At the end of the day, the editor asked you to review the manuscript because of your expertise, not to offer typographical or editorial issues.

Reflect on your tone

Even if the paper has you tearing your hair out in frustration and drawing giant WTFs in the margins, try and keep your tone kind and constructive in your feedback. If anything, you should save your ire for the action editor who sent you the manuscript rather than the original authors, who may be junior and still developing their writing skills, or perhaps writing in their second language. Often, if it was a particularly frustrating review, it may be best to give it a final read with fresh eye before clicking ‘submit’. Part of your task as a reviewer is to encourage scientific excellence, squashing a manuscript rarely will help authors meet this goal.

Be aware of your own biases

We all have those topics that fit within our expertise but that we think are a bit of a research dead end. Or we may have issues with certain labs and are sure this paper comes from them. Focus on the merits of the paper in front of you. Be objective. You don’t have to like the conclusion, and it doesn’t have to be the study that you would have run. It just has to be solid science. If by chance you do not believe you can be objective, this is a reason to abstain from accepting the review.

Be honest about your own limits

If you don’t mention the results section, we’re going to assume you found no issues with it. If in fact, you’ve never encountered the statistical techniques used, we need to know. We can then give more weight to other reviewers’ comments on that section, or get in another opinion. You can use the confidential comments to the editor for this if you don’t want to put it in the comments that go to the author. There is no shame in not being familiar with a particular statistical technique. There are many. If by chance you are unsure if your limits preclude you from reviewing the particular paper, you can always email the action editor to explain the issue.

Don’t make assumptions

This might be a bit obvious, but you don’t know the gender of the lead author, so don’t commit to a gender in the language of your comments. Don’t tell the authors to get the paper proofread by a native speaker as you don’t know that they are not native speakers. Instead, just tell them to get it proofread. Often, you can avoid (or reduce) the use of ‘authors’ and instead refer to the paper/introduction/analyses sections etc.

Don’t be a jerk

Don’t be this reviewer, and don’t be reviewer #2. Instead, be professional and collegiate.

Trust yourself

Remember that you were asked to review this for a reason. If parts of the manuscript don’t make sense to you, they likely won’t make sense to other readers of the journal. It is absolutely appropriate to highlight sections that are not clear to you. Heck, even if it is a complicated statistical analysis, it needs to be explained clearly so the readers can understand. If in doubt, highlight it to the authors.

Be on time, +/- 2 to 4 days standard deviation

Editors understand the busy workload of reviewers and we can understand that, sometimes, reviews will be a few days late. Being more than a few days late (say more than a week) ideally would involve an email to the action editor flagging the delay. We’d rather wait a few extra days and have a good review than a useless or lackluster review that met the deadline. If by chance your review will be delayed by weeks due to an unforeseen circumstance, please let the action editor know. We may decide to move forward with the reviews we currently have to ensure the timely processing of the manuscript.

Reread the manuscript

Often, you may need to reread the manuscript twice or even three times to truly get a good understanding of the study and its limitations. Try and plan for this in your timelines.

Review your review

Before submitting your manuscript, review it. Ensure that it is not merely cosmetic (unless perhaps it is a second or third submission). Ensure it critiques the manuscript (e.g., see the set of questions above) and offer suggestions to address the concerns, when possible. Ensure it is not unduly long by removing repeated points and being more concise in your writing, when possible.

 

Footnotes

1 What’s the point of two-week reminders? No one says: “ooh I’ve only got two-weeks left, I better get on that”. Sexual Abuse, and Archives of Sexual Behavior we’re looking at you.

 

Suggested citation:
Ó Ciardha, C., & Babchishin, K. M. (2018, February 4). Tips for reviewing scientific articles [Weblog post]. Retrieved from https://nextgenforensic.wordpress.com/2018/02/04/tips-for-peer-reviewing-scientific-articles/

Want to submit your own post? Click here to find out how!

Advertisements

One thought on “Tips for peer reviewing scientific articles”

  1. Thanks guys, this is super useful!

    My additional suggestions:

    1) Consider the journal you’re writing for. For higher-impact journals, I expect authors to step up their game in making links to other disciplines, and I’m more likely to ask for additional analyses that I think might be helpful. For lower-impact journals, I’m less picky about introduction sections, for example.

    2) Try to be clear in your language what you think are essential changes that need to be made for the journal to be of publishable quality, versus suggestions that the authors may want to consider (e.g., things you might have done or would be interested to hear more about, but that aren’t necessarily deal-breakers for publication). It is rare for anyone to adopt all reviewer suggestions, so help them out by signalling what you think the most important ones are.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: