May the odds be ever in your favor: a brief comment on the review games in ecology.

Is there a peer-review crisis in ecology? This is critical question that we need to explore to determine whether we should consider introducing changes. Referee selection, number of reviews needed, and the relative importance of referees versus editors in improving the quality of our manuscripts are viable lines of inquiry. There are numerous ways to tackle these questions. To assess the extent that we are in a crisis, I conducted an online survey to calculate the individual decline to review (weighted analysis of requests versus reviews actually done).

I am biased on this topic as I assumed there was a crisis. I also want ecologists to examine the why and how of reviews and this would be a useful indication that we need to so. So, a high decline to review rate would mean that most of us are too busy to review the work of others. And the answer is... in a short editorial on the topic published in Immediate Science Ecology.  It is the article at the bottom of the main page, and the pdf link is to the right of article.  The DOI is still pending, but the article is otherwise final.

In case you just want the punchline, the decline to review rate for ecologists is 49% (+/- 0.02) meaning that about half the requests are declined by the most appropriate referees. This is not a large pool of respondents nor necessarily indicative of all ecologists or what individual journals such as Oikos experience. Nonetheless, I see this a clear signal that we need to add incentives or change handling in some form.

Not included in the article was the importance of gender.  Analyses of productivity and role one serves in the process were however reported. Women accounted for 36% of the respondents, and interestingly, declined to review less often than the male respondents in this dataset. The decline to review rate of men was 1.5 time greater than women. There were differences in the proportions of respondents by gender that participated in peer review in various capacities with 20% of women serving as editors compared to 30% by men. Nonetheless, women do more reviews relative to the number of requests they receive (Figure below), and similar to the relationship reported in the editorial this difference increases with productivity.

So, this is likely a very real cost that women pay in science and is thus a compelling argument for ensuring that we consider managing the diversity of referees we choose to use in reviewing the work of others - not by gender but by career stage (junior versus senior) or by productivity in a specific field. After all, peer review is also an opportunity for collaboration and for building networks for practicing science. In summary, whether we agree on what constitutes a crisis or not, there is an opportunity here for journals in general, and for Oikos in particular, to consider how we select referees and whether it is meaningful to make these criteria transparent and ensure that reviews are spread between junior-senior, referee-editor, or highly productive veteran-newcomer to the games.

Gender redux, slight nuance in the race to publish. I was thinking about the comment posted by JEB on the meaning of the fit lines. Increasing productivity generates a greater divergence between requests and reviews likely due to a visibility effect, i.e. those that publish more papers also receive more solicitations.  Both genders demonstrate this effect, but I was wondering if it an'important' difference (i.e. red versus blue line, like Tron light cycle races when they crash). I did a t-test for each gender to examine the scale of difference between the fit lines of requests-reviews (incidentally the r2 values are around 0.4). The two fit lines were significantly different for each gender - statistically speaking (t-test for females, t = -2.3, p = 0.02; t-test for males, t = -5.8, p = 0.0001), but the mean extent of divergence between the fit models is double that for men relative to women. This corresponds nicely to what your eye can see instantly in the plots that the lines are more different for men. However, the nuance I was considering is whether the female lines in predicted reviews catch up or overlap with males if the productivity scales fully overlapped, i.e. if additional female respondents completed the survey and reported more publications per year. They would not (t-test, t = -4, p = 0.0001) - women that publish more papers would still do more reviews. Fascinating! Disclaimer, this is only a dataset of 257, so I am purely speculating here.