What are meaningful rewards for doing peer review?

4
116112
Posted by Brian Cody, community karma 116112

I'm looking for data on positive experiences scholars have had (or would like to have) for doing peer review work.

I want to think of ways to improve peer review in Scholastica, and in my own scholarly pursuits.

I've been thinking about my own experience, both formally with journals and informally with graduate writing groups, and I identified a few vibrant examples of when I felt truly rewarded during the peer review process:

  1. Getting feedback on my own work. The litany of comments and suggestions always feel to me like an unexpected reward, like someone worked on my article for me.
  2. Comparing my comments to other reviewers, and feeling satisfied at the relative quality of my comments.
  3. Receiving thanks from the person/journal receiving the comments. In identifying this experience, I feel bad at all the times I have NOT thanked people.
  4. In a writing group, being asked to say more about my comment or suggestion. The request might be motivated by a lack of clarity, but I like to think the request signals an interest and respect for my comment.
  5. Being asked out of the blue to review something because the author remembers previous comments I gave that they considered helpful.
  6. Hearing that someone told my advisor that I gave good comments.

 

What rewards do you care about? What makes, or would make, participating peer review a more rewarding experience?

almost 9 years ago

2 Comments

4
549
Gordon Douglas, community karma 549

Participation in the peer review process as a reviewer can indeed feel extremely rewarding or extremely time-consuming and thankless, and everything in between.

In my personal experience, honestly I think I have felt the greatest "reward" when (a) the manuscript I was asked to review seemed interesting and quality in the first place and (b) I was able to see it improved and on to successful publication in the appropriate place. Obviously this is a bit circular, and surely it shouldn't only feel rewarding to serve as a reviewer when the paper is already good and bound for publication from the start.   But again, the times I think of when reviewing has felt tedioius and least rewarding are when the paper pretty clearly is not up to the standards of the journal in question and I feel I'm simply being asked to spend my time just validating what the editors probably already know - that they will decline to publish it.

In identifying this though, I can see that even in the latter cases there is some room to feel like I'm helping - by suggesting areas the paper could be improved, or sometimes by explicitly suggesting more appropriate journals that the author might try submitting to instead (journal-fit is something I'm pretty interested in, so I enjoy that). Regarding the sort of rewards you identified in writing groups Brian, I have to say I don't think I've EVER received specific feedback (positive or negative) on the actual quality of my comments from an editor or an author during peer review for a journal, but I do definitely appreciate this during informal reviewing for friends and colleagues, and I could see that making it feel more rewarding for a journal.

I also like reviewing for a journal that I'm specifically interested in topically (e.g. a specialty journal, perhaps in a field I'm interested in being a contributing and respected 'part' of).  Finally, to some small degree, I suppose one could say that it's "rewarding" in a base sense simply to be able to add 'reviewed for X, Y, and Z journals' to one's CV. This isn't very good for the journal though, as it suggests no cumulative reward for reviewing for the same journal again...

Interestingly, my basic feelings about this apparently seem to match decently with what other reviewers say. According to this 2008 study (see pp. 8-9), most reviewers say they do so for 'altruistic' reasons (including being able to improve a paper and be part of a community), and less for their own gain. However, when material rewards are offered, it DOES apparently increase a prospective reviewer's likelihood of agreeing to review! These are things I've never even thought about (I'd never even heard of reveiwers getting anything for their work in my field, ha ha), so perhaps worth quoting:

"From the reviewers’ perspective, the incentives they said were most likely to encourage them to act for a journal were:

  • a free subscription to the journal (56% said this would make them more likely to review forthe journal)
  • acknowledgement in the journal (44%)
  • payment in kind by the journal, for example waiver of colour or other publication charges, free offprints, etc. (43%).

"Reviewers were divided on whetherthey should be paid for each review they completed: 35% agreed that they should, while 40% disagreed. Those from the Anglophone regions were the most opposed to payment, whereas researchers from Asia and from Europe were on balance just in favour (44% for, 32% against)."

I have to admit I would totally be more likely to review for a journal if it got me a free subscription.

In sum though, that study finds that "For the most part, respondents’ views on these questions appear to be personal matters, independent of their field of research." Huh.

over 8 years ago
login to leave comment
3
227
Will Hauser, community karma 227

Gordon made some great points there.  On the one hand, it is nice to have a journal approach you for a review, especially if it is one at the 'top' of your field.  I do agree with Gordon that it is sometimes insulting if the article they send you for review is one that probably should have been deflected by the editor in the first place.  If the journal considers its reviewers as a valuable natural resource worthy of conservation (as they should) then they ought not to waste that resource on the drudgery of ripping asunder an obviously flawed work.  Besides, on the occassions where I've had to to this I take no joy in it, no matter how gentle I might be.

 

There are some rewards inherent in critiquing that deserve mention.  It sharpens the mental faculties generally (including writing capacity), and particularly around the issue under review.  For a topic I am familiar with, I know exactly what analytical issues must be attended to and if I am not familiar with that topic then careful critiquing is a great way to discover those 'thorny issues'. Moreover, this process has, for me at times, opened up new avenues for research which is probably the greatest reward.

 

Would it be nice to be acknowledged by the journal? Sure.  Should I be paid?  Probably not.  Money tends to currupt and sully the process in ways we cannot anticipate.  Besides, other 'monetary-ish' rewards (i.e. free subscription) would be more than adequate.

 

Lastly, I would say that it is nice to see others' reviews of the same work.  As Brian noted, it is truly gratifying to see your reivew touch on issues that others' did not.  More importantly, I think this operates as a check on the the quality of reviews.  I would be more than a little humiliated to find that my review was cursory and lacking depth relative to others.  In fact, while I am opposed to the author's name being known to reviewers and the reviewers' names being known to the author, I think a strong case can be made that reviewers ought to be known to each other.

over 8 years ago
login to leave comment