Please use this identifier to cite or link to this item:
|Title:||Reliability in the assessment of program quality by teaching assistants during code reviews|
|Keywords:||Programming;Code review;Code inspection;Grading;Quality;Assessment;Reliability;Agreement;Consistency|
|Citation:||Proceedings of the 2015 ACM Conference on Innovation and Technology in Computer Science Education , 346-346, (2015)|
|Abstract:||It is of paramount importance that formative feedback is meaningful in order to drive student learning. Achieving this, however, relies upon a clear and constructively aligned model of quality being applied consistently across submissions. This poster presentation raises concerns about the inter-rater reliability of code reviews conducted by teaching assistants in the absence of such a model. Five teaching assistants each reviewed 12 purposely selected programs submitted by introductory programming students. An analysis of their reliability revealed that while teaching assistants were self-consistent, they each assessed code quality in different ways. This suggests a need for standard models of program quality and rubrics, alongside supporting technology, to be used during code reviews to improve the reliability of formative feedback.|
|Appears in Collections:||Dept of Computer Science Research Papers|
Items in BURA are protected by copyright, with all rights reserved, unless otherwise indicated.