Tips for Reviewing Conference Papers
The goal of reviewing conference papers is not simply to select some for presentation, although that is an important function. Another goal is to help authors improve their papers through revision. This is where your service as a reviewer can pay off most. Admittedly it can be a lot of work – and the poorer the paper, the more work it is to suggest improvements. But without constructive comments, the review process becomes a contest for conference slots rather than a process of scholarly collaboration. Here are some suggestions for keeping your critique constructive.
Be specific.
Instead of saying that the literature review is a mess (or some other four-letter word), describe exactly what is wrong with it. Perhaps it’s an unsynthesized enumeration of all the articles that popped up on a database search. Perhaps it overlooks an entire body of knowledge that is critical to the study. Say that instead of labeling it with the four-letter word. “The literature review needs to be reorganized, with articles grouped and discussed according to the medium involved – such as television, newspapers and magazines.”
Maybe many parts of the paper are unclear. Do your best to pinpoint and explain the areas of ambiguity. For example, you could say something like, “The stated purpose of the paper is to examine the relationship between children and television. This needs to be narrowed down greatly for a conference paper-sized study.” Working the material presented in the paper, see if you can identify one or two promising avenues on which the author could focus in a revision.
Have an open mind, or at least reveal your biases.
If you are a diehard quantitative researcher, and you are assigned a qualitative paper to review, and the call for papers welcomes both quantitative and qualitative work, don’t bash the paper simply for its methodology. It’s not fair and it benefits no one. Either assess the paper according to the appropriate qualitative standards, or admit that you are not capable of doing so. At most, you might discuss why the qualitative approach does not provide a suitable answer to the research question. But don’t force the author to ask a different question that can be answered using quantitative methods. Many of us have been frustrated by reviews in which the judge basically told the researcher to do a different study. Work within the author’s goals.
Be tactful.
There is no excuse for not being tactful in your review. It is not time-consuming to avoid sarcasm and harsh language. This doesn’t mean you can’t criticize the paper. It means you should avoid abusing the anonymity you have been granted as a reviewer. The anonymity allows you to be frank, not rude.
One way to avoid personal animosity is to focus your comments on the paper, not the person. It’s the work product you’re evaluating, not the individual. Use phrases such as “the paper needs” or “the paper would be improved by” rather than framing your comments in terms of the author’s mistakes in writing the paper (“the author did a pathetic job on the discussion section.”)
Write as many comments as your time permits.
Authors are frustrated when they receive wide variation in their reviews. If you have simply circled numbers in evaluating the paper (such as “1 for highly disagree”), the author is left baffled when the judges give divergent scores. If you write a few comments, the author can look for common concerns voiced by judges, or at least understand where the outlier is coming from.
Admittedly, judges don’t get a lot of time to read and critique conference papers under the procedures currently used by most divisions. Rest assured that divisions are considering electronic submission procedures that would give reviewers more time with the papers.
Remember, you can be polite but frank about the paper without recommending it for presentation at the conference. A scathing review is unlikely to build character. Instead, it can drive away promising new scholars by giving the impression that we are a bunch of curmudgeons who would make miserable colleagues. Talented people don’t have to settle for a work environment they find toxic. Let’s make them eager to join us.
By Karen Markin, Rhode Island
Return to AEJMC Officer Resources