In preparation for this week’s Peer Review Week theme of “Quality in Peer Review”, I decided to reach out to several members of our TAA community for insight into the peer review process from either the author’s perspective, reviewer’s perspective, or both.

Regardless of the perspective, I asked for the answer to a single question, “What makes for a quality peer review process?” Eight TAA members share their insights:

Julie Vorholt, Instructor of Academic English Studies at Lewis & Clark College:

“Whether the author or the reviewer, a quality peer review process centers around transparency grounded in communication that occurs in advance of the submission of materials. For example, it’s helpful to have a schedule for the process developed and distributed in advance. The schedules may be slightly different because the author may need fewer details than the reviewer. However, the importance lies in having a clear series of steps to follow. It’s also helpful to have a rubric or another evaluative tool shared in advance of the submission deadline. This allows the author to decide to apply or not and to apply with the materials in their current condition or with needed revisions. Having this evaluative tool supports the reviewer, too, who is more likely to receive materials that match the expectations for the publication.”

Dominique Chlup, founder of Inspiring the Creative Within^® LLC:

“While ultimately the responsibility lies with the authors to demonstrate rigor and accuracy and research reproducibility as a part of the scholarly process, a quality peer review process can help make the article the best it can be when a professional, fair, unbiased, and balanced assessment is provided. Reviewers who take the time to read papers in depth (and to read them repeatedly) provide a service to authors that strengthen papers. For instance, I do not say “yes” to reviewing a manuscript unless I can spend considerable time on it. A busy reviewer may only give a cursory read to a paper and miss subtle and more nuanced mistakes.  I want to be able to read it multiple times and to have the leisure and energy to not just read and review a manuscript but also to fact check and provide additional resources and suggestions for references that the authors may wish to consult to strengthen their work.  A quality review process provides a quality control filter that both checks and improves the quality of the submitted work.

Even when I provide a “reject” review, I want to provide lots of valuable suggestions for how the author(s) can rewrite or reconceptualize the manuscript so that they may ultimately have success placing the paper somewhere else. A quality review process seeks to strengthen the work, not to denigrate the work or disempower authors. In this way, a quality review process can also contribute to a successful review process.”

Tara Gray, Director of Teaching Academy and Associate Professor of Criminal Justice at New Mexico State University:

“As a reviewer, allow at least two hours for your review and type at least one page responding to the prompts from the journal. Try not to make assertions without giving examples. Instead of merely stating that this seems disorganized, give an example of disorganization. Ask questions rather than making statements. Instead of stating this is unclear, ask if it would be clearer if such and such were done? Begin by signing your review even if you take your signature off before mailing it. You want to write in a tone that you wouldn’t mind the author knowing who you are.”

Anna Russakoff, Ph.D., Associate Professor & Chair, Art History & Fine Arts at American University of Paris:

“I’d say that the #1 criterion for making a quality peer review is assuring anonymity. Both reviewers and authors sometimes wind up identifying themselves either in their comments or in their articles (especially in footnotes), and then it seems that the process is not “blind” and objective like it is supposed to be. (In my field, authors and reviewers often know each other.)

For a second point, I would say that the most helpful reviews are the ones that are the most specific. If the article is accepted, or accepted contingent on revisions, the author needs to know exactly which points to address in the final version. If the article is not accepted, detailed comments are even more invaluable to help the author reformulate his/her argument, to push them to take a new angle or address different research questions, and the comments could ultimately help the author find a new venue in which to publish their work.”

Wayne C. Wolsey, Professor Emeritus of Chemistry at Macalester College:

“It’s important that the reviewer have some knowledge of the field which the book covers. One should read the preface carefully, examine the table of contents, look at the references to see if there are any within a year or so of the publication date. Also, see if you can find a published review of the book, such as in the Journal of Chemical Education. Not as useful are ‘reviews’ or comments which one can often find on the Amazon website.

You need to get a feel for the level of writing of the author. How clearly does he/she explain things?”

Jean Marie Carey, PhD, Training Resources Editor, Humanities and Social Sciences Online Material Culture and Vernacular Landscapes and Artifact Preservation:

“From the perspective of being the person undergoing peer review, my first and really only response is: it is time to do away with blind peer review. In the contracting field of the humanities, particularly in unfashionable subjects (such as my own) there really are only a handful of people capable of doing such a review, and of them, an even smaller number will be available to do the actual work. Therefore at least in my experience it is often possible to simply guess who the reviewers are, either from their topical comments, or, if it is a scholar who I have read (and in several cases it has been) from his or her writing style. I recently had someone whose identity I had suspected contact me outside the parameters of the review with additional comments and suggestions, which were very helpful, so I suspect others share my opinion on this matter. 

Particularly in the hard humanities, where we are being crushed from both sides – from STEM on the one and the soft social sciences on the other – it would be tremendously beneficial to junior scholars to receive direct feedback from those who are advanced in the field. Beyond making necessary connections it would help to preserve our intellectual and academic lineage, at least for a little while longer. 

Revealing the identities of the reviewers also by necessity (one hopes) curbs nastiness and identifies potential internecine political and personal conflicts.”

Helen H. Gordon, Academia.edu:

“In writing for an academic journal, I have had some disheartening experiences. For example, I wrote a well-researched paper for the International Rose-Cross journal about the influence of Rosicrucian symbolism in the works of Shakespeare. I explained that ‘William Shakespeare’ is a pen name, probably used by Edward de Vere, 17th Earl of Oxford, who had connections with Rosicrucian and Freemason movements in Elizabethan England. A reviewer who wrote only in French chastised me for not doing more research on Francis Bacon and citing more articles about him. I believe Bacon did help get the Sonnets published, but I’m sure he didn’t write them. There are many Baconians among Rosicrucians, but they are supposed to have open minds.  The reviewer seemed to expect me to conform to his biases — in short, he wanted me to write the paper HE would have written.

I also had a bad experience writing for an Oxfordian journal, although I had published with them before, and the editor had invited me to submit the article. But a couple of the committee members had already made up their minds that Queen Elizabeth was a virgin and that Shakespeare was gay. Their biases interfered with their judgment.”

Laura E. Berk, Ph.D, Distinguished Professor Emerita, Psychology, Illinois State University:

“Here’s a link to a piece that I would recommend on this topic, written by Robert Sternberg in 2003. He addresses the problem in peer reviewing of what he calls a ‘savage review.’ As he notes, it largely emanates from anonymity in the review process, though there are other contributing factors.

There are important reasons for anonymity in reviewing, but a ‘savage review’ is never acceptable. Prof. Sternberg discusses some steps recipients of a ‘savage review’ can take to overcome that unfortunate experience.

The take-away lesson for reviewers is that in critiquing manuscripts, their job is to help other researchers and authors improve at what they do, not attack them. Reviews ought to be crafted as teaching moments for recipients, not devastating moments.”

---

Eric Schmieder is the Membership Marketing Manager for TAA. He has taught computer technology concepts to curriculum, continuing education, and corporate training students since 2001. A lifelong learner, teacher, and textbook author, Eric seeks to use technology in ways that improve results in his daily processes and in the lives of those he serves. His latest textbook, Web, Database, and Programming: A foundational approach to data-driven application development using HTML, CSS, JavaScript, jQuery, MySQL, and PHP, First Edition, is available now through Sentia Publishing.