We often guard zealously against accepting a paper that may not be perfectly right, but in doing so, we often fail to recognize the papers that include truly novel and innovative ideas. It is nearly impossible to reduce the type I error without simultaneously increasing the type II error. This thinking reflects another important issue. By adopting a null hypothesis that a paper is wrong — meaning that a type I error would lead to the acceptance of an erroneous paper — we do our authors a disservice. Our null hypothesis should be that there is something of value in every paper and that the role of the editorial process is to find those nuggets of value. This, too, is advice that Bob gave me a decade and a half ago.

Reviewing papers for publication is subject to all the frailties and flaws of any inherently human process. To paraphrase Supreme Court Justice Potter Stewart, "I can't define what makes for a good paper, but I think I know it when I see it." I will leave it to others to draw additional parallels between published papers and the original topic of Justice Stewart's remarks. I have heard that the elite Israeli units take 90 percent of their recruits based on rigorous physical, mental and emotional testing, but that the other 10 percent are taken at random from among those who would otherwise be rejected. The rationale for this is that the testing process is imperfect; by taking some randomly from the "reject" group, the military gives itself a second chance at these people. Those who are not suitable will wash out very quickly; those with the ability to survive the rigors of the unit will remain and may become exceptional leaders. While we should not knowingly accept papers that are wrong, the reputations of our journals depend more on the exceptionally good papers than on the weak papers that are quickly forgotten.

With these thoughts in mind, we can improve the review process of our papers in a number of ways. Our goal should be to publish exciting and innovative research. With 11 print journals publishing more than 7,700 pages and nearly 600 papers per year, it is probably too much to ask that every paper be innovative and exciting for all members. However, by expediting the review process, we will: (1) attract more papers in time-sensitive areas such as national security and healthcare, (2) provide better service to our authors, (3) enhance the relevance of the papers being published and (4) make our journals more valuable to our subscribers. Reducing the cycle time reduces our risk of publishing out-of-date material, papers that "everyone" has already seen in an earlier form, as well as the risk of losing time-sensitive areas to other journals and disciplines. Reducing cycle times is a win-win proposition.

Strategies to Reduce Cycle Time


I want to outline three strategies aimed at reducing the cycle time associated with our review process. The first entails identifying the role each individual should play in the process, beginning with the authors. Second, technology can assist us. Finally, I propose a policy change for our editorial boards.

A former colleague of mine started almost all of his papers in the same way. The first three or four paragraphs described the problem in general terms understandable to a broad audience. The next paragraph invariably began, "This paper makes the following N contributions." He then explicitly stated the contributions of the paper. (Never was the contribution 1/15th of his tenure case, though he left Northwestern as a tenured full-professor.) Arrogant, you say? Hardly! By the time the editor had read two pages of the manuscript, he or she knew what the problem was and what the contribution of the paper was. If the editor read beyond that, he or she was implicitly acknowledging that the problem was interesting and the contributions were worth making. My colleague had made the job of the senior editor as easy as possible.

The senior editor, the person to whom the paper is normally sent, should determine if the paper is suitable for the journal. Does the paper conform to the editorial mission of the journal adequately? Editorial missions change over time. What might have been acceptable for Management Science 40 years ago, when we had few specialty journals, might not be appropriate now. Authors should review the current editorial mission statement of a journal before submitting a paper. If the paper is not suitable for the journal, the senior editor should quickly reject the paper. Some of our journals do an excellent job of this. The rejection rate within the first 14 days following submission is about 10 percent, 15 percent and 25 percent at Organization Science, Management Science and Transportation Science, respectively. Such a quick decision enables the authors to send the paper to another journal, without prejudice, in a timely manner.

At the other extreme, a senior editor should feel empowered to accept a paper without additional review. Al Blumstein and Dick Larson's 1969 paper entitled "Models of a Total Criminal Justice System," was immediately accepted by the Operations Research editor because it was "too important" to send to referees. Outstanding papers on national security may fall into this category today. Yes, this is risky, but delaying such papers incurs other, perhaps more significant, risks.

Once the senior editor forwards the paper to a department editor, suitability for the journal should no longer be an issue. The department editor, a specialist in the field of the paper, should then determine if the claimed contributions are worth making. If they are not, a quick rejection should again ensue. If they are worth making, the paper is sent to two or three referees, who determine if the contributions have actually been made.

The Referee's Role


The referee's role is not to write the paper in the way they would want it written. Also, the refereeing process is not democratic. If two referees fail to find a critical flaw in a proof, but the third does, the majority does not rule.

Once the reviews are in hand, the editors should determine which comments the authors must respond to. An editor is not doing his or her job if (s)he merely acts as a mailman, forwarding often-conflicting comments to the authors. How often have we received one set of comments asking that the paper be shortened by 25 percent and another set asking for an expanded literature review, more details on the solution algorithm and enhanced computational results? Editors need to filter referee comments and provide clear guidance to the authors. Bob Herman told me that he would sometimes physically cut out comments that were destructive in nature. I, too, had a pair of scissors readily at hand before I forwarded comments to an author. (This was in the pre-e-mail, pre-Internet, pre-historic era.)

In revising a paper, authors can again expedite the process. First, they should respond in a timely manner. If authors expect reviews within months, they should respond to reviews in a similar amount of time. Journal editors have little patience for authors who take 18 months to revise a paper and then expect reviews within two months. Second, authors should (1) highlight the changes made to the paper, ideally on a copy of the manuscript, and (2) respond to the referee comments. Authors need not act on every comment made by each referee, though they should explain why they have chosen not to make some modifications. By highlighting the changes they make, authors can expedite the review of a revision.

Technology can also assist us. Dick Larson, INFORMS' immediate past president, and I have been encouraging our journals to publish queueing statistics on a regular basis. While some of our journals already publish such information — see the outstanding Web site maintained by Steven Shugan, the EIC of Marketing Science — we hope to have all journals doing so within a year. Supply chain specialists advocate visibility across the supply chain, and we hope that the added visibility that public information provides will improve our processes as well. At Management Science, the adoption of an online manuscript submission system increased the visibility of papers across the editorial staff and resulted in a reduction in the average time to first review from almost 200 days in 2002 to 110 days in 2003, despite a significant increase in the number of submissions. The 95th percentile went from more than a year to less than 300 days in 2003, and to about eight months in 2004. These improvements would not have been possible were it not for the availability of the data, coupled with the dedication of the EIC, Wally Hopp and the rest of the senior editorial team to reduce the cycle times.

Finally, I suggest that our journals adopt a policy of sending a paper back to a referee, at most, once after the initial review. In one case, a paper submitted in September 2001 was accepted with minor revisions less than a year and one revision later. It took five more revisions and more than two-and-a-half years for the paper to be accepted. Were the improvements in this paper really worth delaying its release for almost a thousand days? In my experience, multiple passes through the referees usually indicates an unwillingness of the senior editorial staff to reach a decision.

Let's be willing to take risks, but at the same time, we can reduce the risk of publishing out-of-date, untimely papers by significantly reducing our cycle times. The benefits of reduced cycle times and judicious risk taking will far outweigh the costs.

Table of Contents

OR/MS Today Home Page


OR/MS Today copyright © 2006 by the Institute for Operations Research and the Management Sciences. All rights reserved.


Lionheart Publishing, Inc.
506 Roswell Rd., Suite 220, Marietta, GA 30060 USA
Phone: 770-431-0867 | Fax: 770-432-6969
E-mail: lpi@lionhrtpub.com
URL: http://www.lionhrtpub.com


Web Site © Copyright 2006 by Lionheart Publishing, Inc. All rights reserved.