Keywords

adaptive comparative judgment, engineering design assessment, freshman engineering, problem-based learning

Abstract

This article examines the use of an alternative form of assessment for engineering design projects called adaptive comparative judgment (ACJ). The researchers employed an ACJ tool to evaluate undergraduate engineering student design projects in an effort to examine its’ reliability, validity, and utility in comparison with traditional assessment techniques. The ACJ process employed multiple judges to compare the design artifacts of 16 first-year engineering majors. The authors conducted an analysis of the reliability and validity of the ACJ method compared to the traditional rubric used to evaluate the project and the performance data of each student’s design prototype. For these design artifacts, ACJ demonstrated a strong alignment with traditional assessment methods (rs= 0.79, p< 0.01). Yet, neither ACJ nor traditional assessment results were significantly correlated with the actual performance of the design prototype. Additionally, the findings indicate the amount of time each judge devotes to judging student work using ACJ does not significantly impact the reliability of their assessment.

Original Publication Citation

Bartholomew, S. R., Strimel, G. J., & Jackson, A. (2018). A Comparison of Traditional and Adaptive Comparative Judgment Assessment Techniques for Freshmen Engineering Design Projects. International Journal of Engineering Education, 34(1), 20–33.

Document Type

Peer-Reviewed Article

Publication Date

2018

Permanent URL

http://hdl.lib.byu.edu/1877/8310

Publisher

International Journal of Engineering Education

Language

English

College

Ira A. Fulton College of Engineering and Technology

Department

Technology

University Standing at Time of Publication

Assistant Professor

Download

Share

COinS