[Logistics and Policies] [Project Guide] [Reviewing Guide]

Peer reviews will provide (a) freeform advice for the authors on their study, (b) freeform evaluation of the study according to the rubric criteria, and (c) scoring of the rubric criteria. Both (a) and (b) should be entered in the comment boxes with the rubric. All freeform feedback is expected to be collegial.

It is recommended that peer reviewers prepare their reviews offline in a separate document and then copy the completed review into the Canvas peer review when finished.

Advice

A key part of peer review, for example, the peer review of scientific articles, is reviewers' advice for aspects of the paper that can be improved. For example, this advice can (i) provide connections to topics that the authors may have missed, (ii) can suggest variations in the evaluation methods that might lead to different observations, and (iii) can suggest interesting directions in which the work could be improved. One way to think about this discussion is if the authors were your friends and they showed you their report and asked you for advice on how to improve it.

Advice should be provided in the comments associated with the rubric for which the advice is most related. Rubric details are given below.

Evaluation

Peer reviews should also provide written evaluation the project reports according to the following rubric criteria.

Appropriateness. The conducted project satisfies the requirements described by the project assignment.
Execution Quality. The authors execution of the project is high quality. Implementations are (likely to be) correct. Conclusions are (likely to be) correct.
Novelty. The open-ended part of the project was creative and its conclusions (even if negative) allow for new understanding of topics in the space of the project.
Writing Quality. The project report is well written. The structure roughly follows the recommended structure (with deviations that improve flow and readability). It is easy to understand what was done and what the conclusions are.

This evaluation should be provided in the comments associated with corresponding the rubric criteria.

Scoring

Scoring on on the four criteria above will be conducted on a ten point scale with the following interpretations.

10 -- an exemplary report (should be used sparingly)
8 -- a solid report (good reports should receive this score)
6 -- report does not do everything that it should
2 -- report is mostly insufficient, though some aspects of it go in the right direction
0 -- does not even partially satisfy any evaluation criteria.

Scores between those listed above can be used as well for reports that are borderline between two scores.

Recommended Approach to Scoring: Start with a score of 8, increase to 9 or 10 only if the submission is exceptional. Decrease to 7 or 6 if the solution is mostly correct but has minor issues. Decrease lower if there are major issues.