Another peer review reform: Peerage of Science (UPDATED)

Following on from previous posts on reforming peer review (see here, here, and here), I wanted to note a new peer review service, Peerage of Science. PoS is a private company founded by a trio of Finnish scientists, which combines several proposed peer review reforms into one package and then offers them as a service to participating journals, for a fee. Member scientists, called "peers", can submit mss to the service, which other peers can review (anonymously). Members are obliged to review in appropriate proportion to how much they submit, an element of PoS which echoes PubCreds. Participating journals can see the mss and the reviews, and notify the authors if they want to accept the ms. Authors then can choose their preferred offer of acceptance, an element of PoS which echoes ExpressO. To provide additional incentive for scientists to join (which they can do for free, at the invitation of a current peer), and to perform reviews, peers can score one another's reviews for quality. So if you're a good reviewer, you can build up a good score which, at least in theory, is an objective number that you can put on your cv. There are also plans to publish the best reviews in a commentary-type journal, to provide an additional incentive to review. Incentives for review have of course been widely discussed.

PoS is just getting off the ground. They currently have about 500 peers, many of them Scandinavian ecologists. Our sister journal Ecography is currently the only participating journal.

Neither I nor Oikos endorse PoS (full disclosure: I was invited to become a peer, but declined for personal reasons), but I do find it interesting to think about and so wanted to post on it.

As with other proposed peer review reforms, I think PoS will succeed or fail depending on whether a sufficient number of the "players" (authors/reviewers, journals, publishers) think it's to their benefit to use the service. Any serious proposal to reform peer review either has to respect, or have some plausible way to change, the incentives faced by authors, journals, and publishers. For authors and reviewers, I think the incentive to join depends very much on the number and identity of the participating journals. Prospective peers are going to be asking themselves, "Will joining PoS let me publish more papers, faster, in better journals, than I otherwise could?" Frankly, I don't think the possibility of accumulating a good review score, or of having some of your reviews published in a "journal of peer reviews", is going to do much at the margin to attract those who wouldn't otherwise join. What would be attractive would be the possibility of having many journals "bid" on your mss, based on only one set of reviews. But there may be a catch-22 here, because I'm not sure the service will be attractive to journals unless there are many, many peers submitting and reviewing many, many mss, so that journals feel like they need to be able to tap into that ms stream. This is hard to judge, though, as much depends on the fees that PoS charges. How much is it worth to journals or publishers to essentially outsource their reviewing? PoS also is going to be competing with the "cascading review" services that publishers have begun to offer. Publishers are more than happy to have effectively simultaneous submissions--to their own journals--and to share reviews--among their own journals.

It's also possible that some scientists will be uncomfortable joining PoS because they will feel like they're "working for free" for a private company. But on the other hand, reviewing for any journal except a non-profit journal amounts to "working for free" in the same sense, as does submitting to any for-profit journal that charges author fees. There are complex issues here which need to be unpacked.

UPDATE: Mike Fowler pops up in the comments with some trenchant thoughts, and a link to an excellent post on PoS at his own blog. Mike's clearly thought much harder about PoS than I have!