OR/MS Today - December 1999 - Quantified Contributions

December 1999

Quantified Contributions

By Jonathan Caulkins

Erdos-like numbers may shed light on an OR paper's accessibility and relevance

Some ideas that may not pass the "giggle test" can still be valuable for the conversations they generate. Here's a candidate.

Some of my friends (names withheld to protect the innocent) have confided that they do not read carefully every article in the journals they receive. Others have stopped subscribing altogether. No doubt this is in part because we are all so very busy, but I suspect accessibility and relevance may be issues as well.

Theoretically the review process should not only guarantee technical soundness, but also assess the magnitude of the paper's contribution. Is it worth the opportunity cost of not using scarce journal pages to publish another article? Is it of sufficient value to the journal's audience to make the article worth the effort necessary to read and comprehend it?

Unfortunately there is no analog to a thermometer that can be used to measure a universally agreed upon quantification of a paper's contribution. Instead, we rely primarily on instinct and intuition. Perhaps we can complement that expert judgment with a simple heuristic inspired by the Erdos number.

Paul Erdos was a remarkably prolific mathematician, authoring on the order of 1,500 articles with some 500 co-authors [Hoffman, 1999]. As most readers probably know, one's Erdos number is the length of the chain of co-authorships connecting one with Erdos. Erdos' co-authors have an Erdos number 1. People other than Erdos who have written a joint paper with someone with Erdos number 1, but not with Erdos, have an Erdos number 2, and so on. Having a low Erdos number is a badge of honor.

I do not think publication decisions should be based on authors' Erdos numbers, but we might consider the "audience-number" of the paper, where the "audience-number" is the length of the chain of papers connecting the paper in question to what members of the target audience can be assumed to know and understand. That is, the audience-number would be the minimum number of papers such individuals would have to master in order to understand and appreciate the current paper.

For example, our profession generally and some of our journals specifically claim to inform managerial decision-making. For such journals, we might call the audience-number a "managerial decision-maker number" or "practitioner-number" for short. A self-contained paper that built directly on the core knowledge common to managers would have a practitioner-number of 1. A subsequent paper that refined or extended it would have a practitioner-number of 2, and so on.

What exactly qualifies for a "1" is admittedly more nebulous in this context. There are millions of managers with diverse knowledge sets, whereas there was just one Erdos. So, to get started, assume that all practicing managers have taken a basic introductory course in OR and, furthermore, that they remember and understand everything covered in that text. (This generous assumption will help keep our practitioner-numbers from getting out of hand.) So, a paper that proposes a faster way to solve linear programs would get a practitioner-number of 1. Since our prototypical managers already know what linear programming is, they do not need to read any other papers to understand the value of this new paper. A paper that proved a property of the running time of that linear programming algorithm would have a practitioner-number of 2.

Likewise, the definition of what constitutes a link is a little more nebulous. When calculating an Erdos number, links exist if and only if there is a co-authorship relation, which is well-defined [1]. For the practitioner number, a link would exist if mastering the prerequisite paper provided a sufficient basis for reading and understanding the new paper. Mere citation is not sufficient to establish a link. A highly esoteric paper could not be redeemed by adding Hillier and Lieberman to the references.

Finally, there is the question of multiple paths. Understanding the algorithm used by a paper might require reading a chain of three papers. Understanding the substantive material might require reading a different chain of four papers. If understanding both the algorithm and the substance in moderate detail were necessary to appreciate the paper in question, I would suggest that the paper has an audience number of 4. The audience-number would be the length of the shortest path among all paths central to appreciation of the paper in question.

The purpose of audience-numbers would be to measure relevance. A paper might be very relevant (audience-number of one), but still be unfit for publication, e.g. because it contains errors or does not contain original insights. Conversely, a paper with a high audience-number might still be of great value by virtue of its sheer elegance or the importance of its result to some sub-literature. Audience-numbers would complement existing standards, not replace them.

For many journals it may be useful to define a "researcher-number." The researcher-number would be the minimum number of papers someone with Ph.D. training in OR would need to master in order to connect the present paper to the material covered in the coursework of a typical Ph.D. program in operations research. Note: One needs to trace the connections back to the common core knowledge shared by all Ph.D.s in OR, not just those who have specialized in a particular line of research.

I suspect the researcher-number would be pertinent for people with a masters in OR as well, since the coursework required for a masters degree is often comparable to that required for a Ph.D. — the Ph.D. differing primarily in the magnitude of the thesis work required.

Calculating audience numbers would be burdensome for editors or referees, but the onus could be placed on the authors. Just as operations research requires authors to submit a "statement of contribution," journals could require authors to submit an argument for why a paper's practitioner and/or researcher numbers are what they claim. The referees would merely need to verify the argument.

Audience numbers could form the basis for editorial policy. For example, a journal editor could announce that at least some proportion of papers will have a practitioner-number below 3, and such statements might increase the journal's appeal to practitioners. Likewise ensuring that each issue has several articles with a researcher-number below some cutoff would encourage researchers to view the journal as required reading, not as something merely to glance over in search of selected articles in ones own sub-discipline. Audience numbers could also be used like keywords for guiding electronic searching. Mangers might want to search only for articles with practitioner-numbers below a certain cut-off, or to sort the results of a search by practitioner-number so that the more fundamental articles are read first.

Figuring out whether these Erdos-like numbers are actually feasible and useful would take some work. To test feasibility one would want several people to estimate the numbers for a large set of papers to see if different raters produced reasonably consistent numbers. To test utility one would compare the resulting audience numbers to subjective ratings of the articles' quality, with the ratings produced by both current and potential subscribers and by both practitioners and researchers.

Ideas that do not pass the giggle test may not be worth such effort, but they can still be valuable for the conversations they stimulate. I hope this one stimulates as many interesting conversations for you as it has for me and for those with whom I've shared it to date.

References

Hoffman, Paul, 1999, "The Man Who Loved Only Numbers: The Story of Paul Erdos & the Search for Mathematical Truth," New York: Hyperion.
According to the Erdos Number Project web page (http://www.oakland.edu/~grossman/erdoshp.html), mathematical collaborations that lead to publication is what counts. Not normally included are joint editorships, introductions to books written by others, technical reports, problem sessions, problems posed or solved in problem sections of journals, seminars, very elementary textbooks, books on history, memorial or other tributes, biography, translations, bibliographies, or popular works.

Jonathan Caulkins is professor of Operations Research and Public Policy at Carnegie Mellon University's Heinz School of Public Policy and is on the research staff at RAND's Drug Policy Research Center. Caulkins' research focuses on systems analysis and performance evaluation of public policy problems, particularly those pertaining to drugs, crime and violence.

Table of Contents

OR/MS Today Home Page

Lionheart Publishing, Inc.
506 Roswell Street, Suite 220, Marietta, GA 30060, USA
Phone: 770-431-0867 | Fax: 770-432-6969
E-mail: lpi@lionhrtpub.com
URL: http://www.lionhrtpub.com