Pay Referees Per Mistake Caught?

Error message

  • Deprecated function: The each() function is deprecated. This message will be suppressed on further calls in _menu_load_objects() (line 579 of /var/www/drupal-7.x/includes/menu.inc).
  • Deprecated function: implode(): Passing glue string after array is deprecated. Swap the parameters in drupal_get_feeds() (line 394 of /var/www/drupal-7.x/includes/common.inc).
Published by Anonymous (not verified) on Tue, 12/04/2022 - 9:57pm in

James Stacey Taylor, a professor of philosophy at The College of New Jersey, is concerned about the problem that “scholars are not verifying the accuracy of their sources,” and offers up a solution.

Writing at Times Higher Education, Professor Taylor sees this problem as a rational upshot of the incentive structure of academia:

[C]hecking sources is time-consuming, and if a claim is entrenched then it might be reasonably assumed that others have verified its accuracy. Moreover, checking sources is unlikely to yield much professional benefit. If you discover that the sources cited do support the claim that was made, then you haven’t advanced your research by checking them. If they don’t – well, academic journals have little interest in publishing corrections of exegetical falsehoods. And even if you are able to publish your discovery – perhaps in a publication that merely documents the errors of others – you may earn yourself a reputation as a pedant, rather than as someone who advances the field. So neither authors nor referees will receive external benefit from the time-consuming task of fact-checking.

The solution, then, is to introduce new incentives. Here’s his version of that: “pay referees bounties when they detect errors.” Where’s the money coming from? He says:

these bounties should be paid by the authors in whose manuscripts they were detected – and they should be paid whether or not the manuscripts are accepted for publication. This would provide authors with an incentive to avoid error in the first place.

There would be different payouts for different kinds of errors:

Referees should thus be paid different amounts for detecting different types of error: small bounties for detecting an erroneous bibliographic entry, larger ones for identifying misquotations, with the largest of all being reserved for identifying misrepresentations of primary sources.

He doesn’t go into the details of how this arrangement would operate (perhaps escrow accounts that hold the functional equivalent of security deposits?) but presumably there is some way to work that out. Still, there are other questions such a proposal raises, especially for DN’s readership:

  • Are problems with citations and the accurate representation of source material a problem in philosophy?
  • One problem noted often in philosophy is that relevant sources are overlooked (for example, here, here, and here). Should identifying such overlooked sources also earn a referee a “bounty”?
  • There’s a sense in which a good chunk of philosophy, especially the history of philosophy, is already involved in a kind of identification of “misrepresentations of primary sources.” Presumably that work is not what Taylor aims to incentivize with his proposal, but then what would be a workable conception of “misrepresentation” for philosophy?
  • To what extent do proposals like this represent a shift in the general assumptions we make about scholars and their work? One response to recent academic “hoaxes” involving fake data was that they don’t show much beyond the fact that the academic peer review system operates on assumptions of good faith, and so can be tricked by malicious actors; and since such malicious actors are rare, we don’t need to try to restructure the system with even more burdensome forms of review. Have we become more cynical?

Discussion welcome.