By Margaret Harris
My first experience of being peer reviewed did not begin well. Here’s the opening of the referee’s report:
“The purpose of publication is to disseminate knowledge to other people who may be able to use it. Since the model dramatically fails the authors’ own experimental tests more than half of the time, I can’t imagine anyone wanting to use it. I therefore recommend against its acceptance, here or anywhere else.”
What followed was a cavalcade of sarcastic remarks, incorrect assumptions, and occasional admissions that the referee didn’t know enough about the paper’s topic to make judgements (not that it stopped them). My senior co-authors were incensed; I was devastated. The words “I recommend against its acceptance, here or anywhere else” were burned into my brain and even now, more than 10 years later, reading them feels like a punch in the gut.
Shortly after receiving that scathing report, however, one of my senior co-authors took me aside. It turns out that his first peer-review report had been just as bad: he’d been told “Nothing new is in this paper, and what is new, is wrong” (I paraphrase slightly). That paper, he added, had eventually been published, and had gone on to receive more than 60 citations.
Buoyed by this admission, and guided by the referee’s tiny handful of useful comments (plus a far more productive critique from a fellow student), I set to work. A few months later, we submitted a much-altered version of the paper along with a request that, due to the adversarial tone of the original referee’s report, the new version be sent to someone else. This second referee proved more amenable, praising us for a “well-written manuscript with results that are likely to be of interest to many readers” and noting that while our model was indeed flawed, “any simplified model will have such problems”. In conclusion, the second referee wrote, “I think the authors are describing something that is novel, and of utility, and so I recommend that the manuscript be published.” After a few minor revisions, it was.
So in some ways, I suppose my first peer-review experience was a success. The revisions definitely made our paper better, and I’m pleased to report that, just like my co-author’s first paper, it has since gone on to receive more than 60 citations. Doing the revisions also made me a better physicist. My original contributions to the paper had been limited to running someone else’s code and performing experimental tests of someone else’s model, but by the time we resubmitted, I had a very thorough understanding of every aspect of the work.
But at the same time, it was a pretty miserable experience. I was a new PhD student at the time, and several other not-so-pleasant events in the lab had left me desperately short of self-confidence. If my co-authors hadn’t stepped in to reassure me that the first referee’s tone was out of line, I might have quit. Our experience also demonstrates why relying on the opinion of a single referee is a bad idea; although multiple reports do mean more work for scientists and journal editors, the potential for abuse by a single biased, spiteful or grumpy individual is just too great.
So my question for Peer Review Week is this: aside from mandating multiple referees (as many journals do) what changes could be made that would preserve the positive aspects of my first peer-review experience while eliminating the negatives? Please add your suggestions to the comments – and, if you’re brave enough, share your own first experience of peer review.
I am not surprised at what happened to Dr. Harris first paper. I remember sometime back I submitted a paper to a European journal and what came back from their single referee was just abusive. I submitted the paper to Journal of Applied Physics and they accepted and published it. Thereafter, I sent a copy of the published paper to the journal and asked them to give the copy to their reviewer. I added that it is unprofessional to abuse authors of prospective papers. What I have learned is that there are some European journals whose reviewers will not pass a paper authored by Africans. In fact, I have since learned that a senior South Korean Physicist also came to the conclusion that some European journals just will not accept papers authored by Koreans(Asians). The Korean Physicist is now a reviewer of a number of European journals.
Thanks for your comment, Dr Oyoko. Bias due to authors’ names, countries or institutions is a serious issue, and some journals (including two that are published by Physics World’s parent organization – you can read about them here: http://iopscience.iop.org/page/peer-review-week) have adopted “double-blind” peer review to combat it. In this system neither the authors nor the referees know each other’s names or identifying details. It’s not perfect, because it’s sometimes possible to tell who has written a paper based on who they cite (or, in small fields, what the work is about), but it might have helped you and your South Korean colleague.
Name-based bias can also affect female scientists in peer review, but since I always published using my initials rather than my full name, that won’t have been a factor in this particular case.
It is time that the physics community will start to pay attention not only to sexual-harassment, but also to refereeing-harassment. Sexual-harassment has much sever consequences. Nonetheless, refereeing-harassment is much more common, can also harm the carrier of people, and it is very hard to deal with because of anonymity. Editors should go through a training to know how to deal with refereeing-harassment, as much as there are workshops in working places on preventing sexual harassment.