The multiple-choice test question is a popular item format used for tests ranging from classroom assessments to professional licensure exams. The popularity of this format stems from its administration and scoring efficiencies. The most common multiple-choice format consists of a stem that presents a problem to be solved accompanied by a single correct answer and two, three, or four incorrect answers. A well-constructed item using this format can result in a high quality assessment of an examinee's knowledge, skills and abilities. However, for some complex, higher-order knowledge, skills and abilities, a single correct answer is often insufficient. Test developers tend to avoid using multiple correct answers out of a concern about the increased difficulty and lower discrimination of such items. However, by avoiding the use of multiple correct answers, test constructors may inadvertently create validity concerns resulting from incomplete content coverage and construct irrelevant variance. This study explored an alternative way of implementing multiple-choice questions with two or more correct answers by specifying in each question the number of answers examinees should select instead of using the traditional guideline to select all that apply. This study investigated the performance of three operational exams that use a standard multiple-choice format where the examinees are told how many answers they are to select. The collective statistical performance of multiple-choice items that included more than one answer that is keyed as correct was compared with the performance of traditional single-answer, multiple-choice (SA) items within each exam. The results indicate that the multiple-answer, multiple-choice (MA) items evaluated from these three exams performed at least as well as to the single-answer questions within the same exams.
College and Department
David O. McKay School of Education; Instructional Psychology and Technology
BYU ScholarsArchive Citation
Thayn, Kim Scott, "An Evaluation of Multiple Choice Test Questions Deliberately Designed to Include Multiple Correct Answers" (2010). All Theses and Dissertations. 2450.
multiple-answer, multiple-correct, multiple-choice, number correct, test question