Sunday, January 26, 2014

How Many Papers Should I Review For This Conference?



My field's main conference is the Annual Meeting of the Cognitive Science Society, or, as it's commonly called, "Cognitive Science." People submit 6-page papers. Each one needs to be reviewed by three people for a decision to be made. It's a part of a scholar's civic duty to review papers. It's natural to review papers for the conferences you submit to. But how many should you agree to review?

One way to think about it is that you should be pulling your weight, and not taking advantage of the system. So if every paper needs three reviewers, then maybe you should agree to review 3 papers for every one you submit.

There are two things wrong with this simple calculation--not all papers are single authored. If you're submitting a paper with 6 authors, maybe the pain should be spread out a bit.

The other factor is that there are freeloaders out there, and you might want to do your part to make up for them.

So what I do is say I'm going to review 5 papers for every one I submit. But this is divided by the number of authors I have on each paper-- so it's 5/n for each paper, where n is the number of authors per paper.

This summer Cognitive Science is in Quebec City. Nice and close! So my laboratory is submitting lots of papers. For papers with my name on it got two I'm the sole author on, three with two authors, and one with 6 authors.

So I'm going to review (5/1) + (5/1) + (5/6) + (3 * (5/2)) papers this summer. That's 18.33.

That's a lot of papers, but it's proportional to the amount of work I'm putting on everybody else in the community, so I'm doing my fair share.

It's easy to calculate using wolframalpha.com

Pictured: A charolais in Venezuela. From Wikimedia Commons.

Bookmark and Share

2 comments:

Michael Cox said...

Yes that is quite a few comments. If you review for multiple conferences each year (I did 6 last year), then the review burden is excessive. However I did not submit to every conference I reviewed for. In any case, I think you will be busy this year.

David Landy said...

One very small point: cog sci submissions typically have 3 reviewers plus a "meta-reviewer", who doesn't write a full review, but does essentially the same amount of work, reading each paper, rating them, and producing integrative summaries.

Given that, it's distressing to see in cold numbers how large the burden is per submitted paper. Maybe we should reduce the number of reviews per paper?