How I write reviews

The goal of peer review is to accept papers. But peer reviews are difficult in practice.

I have a pretty specific template I try to stick to, originally inspired by Dan Cosley.

1. Summarize their argument

2-3 sentence paragraph.

Summarize the main point of the piece. Bonus points if you can highlight the intended contribution more crisply than the author - that's gold for the author, and a great exercise.

2. Say what you liked about the paper

One paragraph.

Describe what was strong about the paper - what was good. There's always a nugget of gold in the paper, even if it's just that the paper touches on some interesting subject matter. This is for the author's benefit: it highlights what's special, strong or unique about their work.

3. Describe the major points that need improvement

One paragraph per point.

Get together the main weaknesses of the paper, and devote one paragraph to each. Don't go overboard here; there are never more than three big problems with a paper, and if there are, then there's probably only one problem (that the framing is poor: it isn't well motivated, or isn't well situated in prior work).

4. Describe minor issues

A bullet point list, one point per issue.

Describe minor qualms, gripes, typos, writing issues in a bulletpoitned list.

5 Optional: High-level recommendation

In a peer review context, give a high-level signal about whether you think the paper should be accepted or not. For the AC or editor, this is gold.