[A surgeon-blogger known as Skeptical Scalpel tries to educate readers about the medical journal peer review process. He has been a surgeon for 40 years and was surgical department chairman and residency program director for over 23 of those years. He is board-certified in general surgery and a surgical sub-specialty. He has over 90 publications including peer-reviewed papers, case reports, editorials, letters and book chapters. He is an associate editor of a journal. He has been blogging for a year-and-a half.]
There is a possibly some misunderstanding among science journalists regarding the process that the term “peer-review” encompasses.
I am an associate editor (AE) of a medical journal with a respectable impact factor. I also am or have been a manuscript reviewer for five different journals. I feel qualified to describe how manuscripts are reviewed and published.
In 2012, authors submit a manuscript electronically to the journal. It is assigned to an AE who screens it for appropriateness, format and, on occasion, readability in the English language. Manuscripts are not blinded. AEs and reviewers are aware of the authors’ names and their institutions.
The AE emails prospective peer-reviewers asking if they are willing to review the submission. Reviewers are chosen based on their self-reported areas of interest. They become listed as peer-reviewers by demonstrating expertise, usually having submitted papers of their own. They may also be well-known experts through society memberships or familiarity with the journal’s editorial board members. I once became a peer-reviewer for a journal after writing a letter to the editor pointing out a statistical flaw in a published paper. Although seen by many as a career-enhancing, the jobs of AE and peer-reviewer are not compensated.
If all goes well, the peer-reviewers return their recommendations in a timely way. Unfortunately, being a peer, an expert or an author in a field related to the manuscript’s topic does not necessarily mean that one can review a research paper competently. Most journals have guidelines for reviewers but no way to tell if the reviewer has read them. We often receive “two-sentence” reviews of 25-30 page (double-spaced) manuscripts.
A manuscript would have to be quite extraordinary to elicit only a two-sentence review. The reviewer may have been too busy, disinterested, incapable or not motivated to do a thorough job. But then why would he have accepted the assignment? That’s one of life’s great mysteries. The AE may have to become a peer-reviewer at times.
Assuming the AE receives two or three adequate reviews, he decides to accept, accept with revisions or reject the paper and forwards it to the editor for a final decision.
Here is what we cannot do. We cannot verify that
1. the data are not fabricated;
2. all authors deserve to have their names listed on the paper;
3. no plagiarism has occurred;
4. the paper is not an attempted duplicate publication.
Journals have no resources to investigate any of these issues. We must accept the word of those submitting. Among other causes, pressure on faculty to publish and/or greed may promote scientific misconduct.
Is this a good system? No. What are the alternatives? I don’t know. I pointed out in a previous post that many more people have read my blog than ever read my research publications. One day, every paper may be posted and critiqued by the scientific public, a movement that has already begun on websites such as Faculty of 1000.
Meanwhile, expect to see more publications retracted as internet users discover and expose fraud and duplicate publications. For more on retractions, follow the blog Retraction Watch for interesting insights into the process.
One of the drivers of the proliferation of journals, both online and print, is the requirement of most Residency Review Committees that faculty of residency training programs must engage in research. This rule is not “evidenced-based,” as there is no proof that a surgeon has to do research in order to be a good teacher or role model. Sometimes the opposite is true; the researcher can’t teach at all.
Residents choose to train at community hospitals because they do not want to participate in research. [Anecdotally, I think many residents at university hospitals would rather not do research too.] As is the case with faculty, there is no proof that forcing a resident to do research will result in important discoveries or make her a better surgeon.
Look at this language from the RRC forSurgery.
Some members of the faculty should also demonstrate scholarship by one or more of the following:
II.B.5.b).(1) peer-reviewed funding;
II.B.5.b).(2) publication of original research or review articles in peer-reviewed journals, or chapters in textbooks;
II.B.5.b).(3) publication or presentation of case reports or clinical series at local, regional, or national professional and scientific society meetings
Since I dropped out of the business of training residents, I have been actively blogging and not cranking out mindless publishable research. Here is an interesting fact. I have no doubt that far more people have read what I have written in my blog for a year and a half than ever read all of my 95 published works combined.
For example, I wrote a blog entitled “Statistical vs. Clinical Significance: They Are Not the Same” in August of 2011. To date, it has been viewed 4466 times. I would guess that one post alone has been read by more people than ever have read my combined published papers. I have 1070 followers on Twitter. Again, it is likely that more people have read what I tweet than ever read my scholarly works.
So what’s the point? Although I have written that individuals who participate actively in social media like Twitter have very little influence when one looks at the big picture, the same can be said of publishing a journal article. Who really reads the 25 or so critical care journals that are currently being published online and in print?
Did I have more influence with my published writings or do I have more influence now with my blogging and tweeting? What do you think?
PS: Just like a journal article, I have cited myself three times.
http://skepticalscalpel.blogspot.com/