Learning by evaluating (LbE) through adaptive comparative judgment

Keywords

adaptive comparative judgment, assessment, design-thinking, learning by evaluating, peer assessment

Abstract

Traditional efforts around improving assessment often center on the teacher as the evaluator of work rather than the students. These assessment efforts typically focus on measuring learning rather than stimulating, promoting, or producing learning in students. This paper summarizes a study of a large sample of undergraduate students (n = 550) in an entry-level design-thinking course who engaged with Adaptive Comparative Judgment (ACJ), a form of assessment, as a learning mechanism. Following random assignment into control and treatment sections, students engaged in identical activities with the exception of a 20-minute intervention we call learning by evaluating (LbE). Prior to engaging in a Point Of View (POV) creation activity, treatment group students engaged in LbE by viewing pairs of previously-collected POV statements through ACJ; in each case they viewed two POV statements side-by-side and selected the POV statement they believed was better. Following this experience, students created their own POV statements and then the final POV statements, from both the control and treatment students, were collected and evaluated by instructors using ACJ. In addition, qualitative data consisting of student comments, collected during ACJ comparisons, were coded by the researchers to further explore the potential for the students to use class knowledge while engaging in the LbE review of peer work. Both the quantitative and qualitative data sets were analyzed to investigate the impact of the LbE activity. Consistent with other ACJ research findings, significant positive learning gains were found for students who engaged in the intervention. Researchers also noted that these findings did not indicate the actual quality of the assignments, meaning the while students who engaged in the LbE intervention were better than their peers, they were not necessarily “good” at the assignment themselves. Discussion of these findings and areas for further inquiry are presented.

Original Publication Citation

Bartholomew, S. R.,Mentzer, N., Jones, M., Sherman, D., & Baniya, S. (2020). Learning by Evaluating (LbE) through Adaptive Comparative Judgment. International Journal of Technology and Design Education.

Document Type

Peer-Reviewed Article

Publication Date

2020-11-21

Permanent URL

http://hdl.lib.byu.edu/1877/8265

Publisher

International Journal of Technology and Design Education

Language

English

College

Ira A. Fulton College of Engineering

Department

Technology

University Standing at Time of Publication

Assistant Professor

Link to Full Text

Share

COinS