adaptive comparative judgment, engineering design assessment, freshman engineering, problem-based learning
This article examines the use of an alternative form of assessment for engineering design projects called adaptive comparative judgment (ACJ). The researchers employed an ACJ tool to evaluate undergraduate engineering student design projects in an effort to examine its’ reliability, validity, and utility in comparison with traditional assessment techniques. The ACJ process employed multiple judges to compare the design artifacts of 16 first-year engineering majors. The authors conducted an analysis of the reliability and validity of the ACJ method compared to the traditional rubric used to evaluate the project and the performance data of each student’s design prototype. For these design artifacts, ACJ demonstrated a strong alignment with traditional assessment methods (rs= 0.79, p< 0.01). Yet, neither ACJ nor traditional assessment results were significantly correlated with the actual performance of the design prototype. Additionally, the findings indicate the amount of time each judge devotes to judging student work using ACJ does not significantly impact the reliability of their assessment.
Original Publication Citation
Bartholomew, S. R., Strimel, G. J., & Jackson, A. (2018). A Comparison of Traditional and Adaptive Comparative Judgment Assessment Techniques for Freshmen Engineering Design Projects. International Journal of Engineering Education, 34(1), 20–33.
BYU ScholarsArchive Citation
Bartholomew, Scott R.; Strimel, Greg J.; and Jackson, Andrew, "A Comparison of Traditional and Adaptive Comparative Judgment Assessment Techniques for Freshmen Engineering Design Projects" (2018). Faculty Publications. 5578.
International Journal of Engineering Education
Ira A. Fulton College of Engineering and Technology
© 2018 TEMPUS Publications.
Copyright Use Information