Opening the Doors of Anonymity

ANDREA DENNIS

Peer reviewing has been an integral part of the editing and publishing process since scholarly journals were established and is used as a form of regulation by scholarly journals spanning many fields. The traditional method of using scholars who are anonymous to the author for the critique is employed to ensure quality, improve the presentation, and offer credibility. All scholars know that they must undergo the process of having their article peer reviewed if they would like it to be printed in a decent journal and many of them are welcome to the assistance that it provides. This collaborative process allows an author to reach their full potential within their article with the help of other experts in their field. Essentially, the process of peer review is literature’s way of adopting its own scientific method, “principles and procedures for the systematic pursuit of knowledge involving the recognition and formulation of a problem, the collection of data through observation and experiment, and the formulation and testing of hypotheses.” [1] The authors present their hypothesis to the reviewers and they, in turn, decide if it is a valid theory worthy of publication in a scholarly journal. Recently, some people have begun to question the stringency of the process and whether or not it should be opened to become more of a dialogue between contributors instead of shut away with such strict confidentiality. What I found to be interesting about Howard’s article was the hesitation shown by everyone involved in the testing of open reviewing when it seems to be a fantastic way to streamline the peer editing process. I think that they should have been more confident in their abilities and in the new process. And I believe that an open peer reviewing system is going to become the new norm because it is a phenomenal process that allows many individuals to have a voice.

As students, we have been told time and time again that we must only use articles that have been peer reviewed because those that have not gone through the traditional process are not deemed credible enough for any of their information to be of use to us. Every single time that I have done research for a paper I have always held in my mind the rule that if an article has not been peer reviewed then it is not worth my time. It is now an automatic function for me to check the “Peer Reviewed” box on all of the search engines while I am looking for articles that support my own theory.

Scholarly journals are all essentially required to employ some form of the peer review process; most of them allow only forms that are at least partially blind because it is believed that this provides a more credible review. A “double blind” review is the most common in the scientific community; in this form the reviewer and the author both remain anonymous to the other. The “single” review is the form that many literature journals have adopted, and what we are doing for our own editing assignments, where the identity of the reviewer is not disclosed to the author but the reviewer is aware of the authors’ name.

Generally a journal searches out individuals of similar competence to the producers of the submitted works to participate in the process; but one such journal decided to try a very different strategy for one of their issues. The Shakespeare Quarterly that is sponsored by the Folger Shakespeare Library chose to test out the new concept of the open review for one of their issues with varying degrees of feedback from both reviewers and authors on the experiment. The articles that were chosen for the trial were posted online in their original draft and opened for other scholars to read and critique as they wished. [2] The transparency of the project was its most striking quality for me; not only were the authors’ names shared with the reviewers, but the reviewers’ names were placed with every single comment that they made in the article. I would think that knowing their names were going to appear alongside their comments and edits would have made the reviewers take a little more time and put a little more thought into what they were saying. It seems, from the feedback that the journal got from the reviewers and authors, to have forced more accountability onto those peers who gave their input.

Although the process is generally considered to be essential to the quality of academic work, peer review has been criticized as ineffective, and slow. Richard Horton, the editor of the British medical journal The Lancet[3], said: “The mistake, of course, is to have thought that peer review was any more than just a crude means of discovering the acceptability – not the validity – of a new finding. Editors and scientists alike insist on the pivotal importance of peer review. We portray peer review to the public as a quasi-sacred process that helps to make science our most objective truth teller. But we know that the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong.” [4] I personally find his view a tad too harsh; there are many ways that the peer reviewing process can enhance the quality of articles, though the new development of “open peer review” could be a better fit, at least in the world of literature.

One of the criticisms of the open review system, and probably a big reason as to why it has not been attempted by legitimate journals in the past, is because scholars are concerned with the quality of the comments and edits made by those who may not have the right qualifications to comment on an article. This was a concern of many of the authors whose articles were chosen to be a part of this experiment, that their article would no longer be considered as professional because it may not get reviewed by the correct people. I was surprised myself when they found that the open process was just as controlled as the traditional review and that many of the academic hierarchies were kept in place. I would have expected people to have been so excited about the possibility of contributing to a journal that they would jump at the opportunity even if they knew that they weren’t qualified. But those junior scholars who wanted to contribute contacted the author offline because they did not want to post their comments online in case the contradicted with the senior scholars which seemed like a very polite way to go about it.

Another concern that was brought up was the question of how to ensure that the journals are not catering to what the masses enjoy as opposed to legitimate scholarly work. If a paper goes online and people are bored or uninterested in the topic they may leave bad reviews or completely pass over it for another, more interesting one. I believe that the form in which these papers are set up online ensures that they get the proper responses from the proper individuals who would take an interest in any scholarly topic. Even if a paper topic is seen as boring to some doesn’t mean that it should not be published. Another question to counter this objection is how can we be sure that this is not already happening in our traditional peer review process. If a paper is only sent to a few scholars to be reviewed and one finds it uninteresting and gives negative comments the paper may not make it into the journal because of an individuals’ bias. This may happen regardless, but if articles were posted in a public forum they have a better chance at getting reviewed by the right people.

I found that in the scientific community there have been some journals that have begun to make changes of their own just like the Shakespeare Quarterly. One of the British medical journals has required that all reviewers must sign their reviews and that all of the comments made on the article must go to the author, much like the open review process that the Shakespeare Quarterly attempted. The argument now is whether or not assigning the reviewers names to the articles promotes accountability or if the anonymity helps maintain their objectivity as they read through prospective journal articles. Horton states that “Journals are not anonymous, he argues, and neither are people who write letters to the editor, so why are peer reviewers? I think it’ll be as quaint in 20 years’ time to have anonymous reviewers as it would be to send anonymous letters to the editor.”[7] Personally I am inclined to agree with him.

The process of an open review system offers much more insight into an article or essay that the author may not have had access to when under the double- or single-blind system that is normally used in scholarly institutions.  I would compare the open peer review system to a conference setting. The papers are not necessarily finished or polished but the authors bring them before others in order to get more insight into what they are trying to convey. Comments are given and questions are asked which make the author take a closer look at their work, which in turn makes a better final product. The open peer review process is much like an online version of a scholarly conference. I think that the open form of reviewing gives a more honest view as to why an article is accepted or rejected and having it be available for more scholars to contribute means that more people are working to make that article as good as it can possibly be. A larger number of contributors means more opportunities for improvement and healthy debate; we in the academic world could literally watch the entire process unfold with this kind of transparency which I believe would be beneficial for everyone.

[1] Merriam-Webster Dictionary, 3rd ed. s.v. “Scientific Method.” http://www.merriam-webster.com/dictionary/scientific%20method

[2] Jennifer Howard, “Leading Humanities Journal Tries ‘Open’ Peer Review, and Likes It”, http://web.a.ebscohost.com/ehost/detail/detail?sid=774c95f2-4a12-4092-8e13-888c58220632%40sessionmgr4003&vid=0&hid=4101&bdata=JnNpdGU9ZWhvc3QtbGl2ZQ%3d%3d#db=aph&AN=53020369

[3] http://www.thelancet.com/

[4] Alison McCook, “Is Peer Review Broken?”, accessed March 3, 2015, http://www.the-scientist.com/?articles.view/articleNo/23672/title/Is-Peer-Review-Broken-/

[5] Peer Review Congress, http://www.peerreviewcongress.org/index.html

[6] At least one participant pointed out that the humanities’ subjective, conversational tendencies may make them well suited to open review–better suited, perhaps, than the sciences.

[7] McCook

Advertisements

4 Comments

Add yours →

  1. Hi!

    I love your post. Great insight and research. I am particularly interested in open review because of the unique opportunity it gives to burgeoning scholars. Having access to at least see the original articles, edits, and review process is invaluable for a young scholar’s development as a writer, editor, and thinker. I would not be surprised if many young scholars, after reading the edits made my advanced scholars, went back to edit their own work with new ways of thinking in mind. Rather than viewing academia as a cold, stuffy, and intimidating world, young scholars may ease their way into this world, seeing the active, encouraging, and critical community at work. In this, I think open access may be one small step away from focusing on the prestige of publishing and one small step toward focusing on the sparking of ideas, of wonder, of community, and of awe in all scholars.

    Like

  2. Andrea,
    Thank you for your commentary debating the use of open review versus an anonymous review system. I agree with you on many points. Firstly, I agree that reviewers who sign their comments will scrutinize both substance and tone of their feedback since they have their own reputations to consider. Additionally, this would offer further dialogue between the author and reviewer. If the author wanted to query a reviewer, it would be impossible via old-school methods, unless the editor of the publication were to act as liaison.
    Listening to an alumnus recount his experiences in submitting papers for publication, I felt that at times it was unfortunate there were only two reviewers providing feedback on his work. It seems that after working hard to produce and submit something for publication, that receiving a larger body of constructive feedback would not only be more helpful, but it seems more time-efficient. As you mentioned, the open review experience would feel more like a conference setting. That influx of feedback, perhaps coming from different perspectives such as age, experience, creative ideas, and differences of career specialization, could offer a more heterogeneous discussion. And this may lead to others commenting on reviewer feedback, taking ideas to even the next level.
    Thank you for including the information about the International Congress on Peer Review and Biomedical Publication. Since writing and publishing work is so critical to the scholarly fields, it makes perfect sense that this topic would be worthy of its own congress, and it would seem this would be an excellent platform to discuss changes to improve the processes involved in scholarly publication.
    Thanks for your blog, Andrea!

    Like

  3. Hi Andrea! Great post—I am glad that you point out the notion that editing is a “collaborative process [which] allows the author to reach their full potential,” because it seems very important to remember that the author’s text is unlikely to be well-received in the scholarly community (or to be as well-written) without interacting with others and seeking constructive feedback through the review process. You distinguish various forms in which this collaboration can occur, such as single- and double-blind reviews and, of course, the open peer review process. Your discussion raises some interesting questions about which of these interactions best facilitates the development of excellent work; does everyone get along better when the whole process is anonymous, or can the scholarly conversation be just as productive (and still be kept manageable and under control) with the open review process?

    You mention that the open review experiment at The Shakespeare Quarterly seemed to result in a heightened sense of “accountability [for] those peers who gave their input,” and although these contributors were probably more careful not to let personal biases emerge in their critiques (so that they would not appear petty to other reviewers and to the author), I wonder if their language was also more tempered and reluctant to give glaring—but constructive—criticism. It is possible that the open reviewers would have been more complimentary and less critical in order to avoid potentially jeopardizing professional relationships with the author or other reviewers who may disagree with their criticisms. The transparency of open review seems to carry the potential of reducing the transparency of reviewers’ comments, potentially restricting the extent to which the author will revise the work to reach its “full potential.” Yet I agree that the open review process seems like an excellent opportunity for scholars to initiate a continuing conversation that furnishes ideas for improving their writing. The open review process also affords authors the opportunity to know and consult their reviewers on future projects, such that the open review process could certainly be a great fit for enhancing “the quality of articles,” as per your argument. I am not sure how to resolve the issues in transparency that seem to come along with open review (including the unwillingness of junior scholars to rescind their anonymity and submit their reviews where better-established scholars can see them), but it does seem that open review can prove useful and effective as a quicker and more ongoing conduit for textual improvement than the traditional peer review process.

    Like

  4. Jenny Bunkers May 14, 2015 — 7:51 pm

    Hi Andrea, I really enjoyed your presentation paper on the peer-review process. The article interested me as well and I do believe that this is a question that will eventually be answered with a different review system than what is currently used for scholarly sites. I believe this because as all things concerning most any academic field with all the pertinent and timely questions concerning this issue will likely not go away. I agree with you about the open peer review process being a good thing for scholars, yet I think it should be limited to those with qualifications to comment because of their background. I did think that this way of reviewing should be open to everyone at first thought, but I think that could get a little out of hand with the potential enormity of responses and some that truly do not add any value to the process for the author. With the current (and I am sure future) technological advances which make information sharing easier than in the past the open peer review system would be an easy thing to accomplish and incorporate into scholarship. Do you keep reviewers anonymous? I think so, or at least with the opportunity of writing an anonymous response. The review can go many ways and provide too much influence on the writer with any bias they may have with a particular reviewer or a disregarding of certain ones due to a myriad of reasons.
    It has never occurred to me to check a peer review box while researching through different scholarly sites. I think that might be something I will check into, but I also believe that peer review may have too much influence on the work and potentially hurt the author’s main ideas. The whole idea of heavy editing and other peer’s comments and concerns could turn into something overwhelming. Of course, the opposite way of thinking about it is getting out of your own head and truly seeing (hopefully) all aspects of the work. I do enjoy feedback on my writing and it has proved invaluable, but I am also aware that if you only have our current system then there is a much higher chance of not truly getting the benefit. I feel confident that open peer review is a positive thing, yet I would like to see limitations, such as the right to show work or review work anonymously. This subject is very thought provoking and I am sure we will see more and more changes to our current system.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: