ASSESSMENT OF RUBRIC-BASED EVALUATION BY NONPARAMETRIC MULTIPLE COMPARISONS IN FIRST-YEAR EDUCATION IN A JAPANESE UNIVERSITY

Abstract

The rubrics have become a widely referenced and utilized form of assessment on campuses across internationally. But rubric can be an asset in any classroom and at any education level but it needs to be implemented correctly. Our research question in this study is whether students were evaluated consistently and equally from teacher to teacher using rubric. To answer this research question, we performed statistical estimation using nonparametric multiple comparisons. This article reports on a normalizing rubric evaluation by nonparametric multiple comparisons in a first-year course called “Manaburu I” offered at Kobe Tokiwa University. “Manaburu” is a word coined by us: “manabu” ‘learn’ in Japanese + English able. Thus, “Manaburu” refers to Self-Directed Learning I. In the course, about 20 teachers teach about 350 students (16–17 students per teacher). Students are organized into groups of about 6. It is of course difficult for 20 teachers to evaluate their students consistently among them, making this course an appropriate site for the evaluation. We constructed a rubric for the course, under which teachers were meant to evaluate students, and presented it to both teachers and students. Our research question was whether teachers evaluated students consistently and equally according to the Steel–Dwass estimation method, a strict statistical estimation method for nonparametric multiple comparisons. The results show that teachers do not evaluate students equally. Suggestions for future research, more attention to validity and reliability, a closer focus on learning and research on rubric use in higher education.