Abstract

The resources required to have humans score extended written response items in English language learner (ELL) contexts has caused automated essay scoring (AES) to emerge as a desired alternative. However, these systems often rely heavily on indirect proxies of writing quality such as word, sentence, and essay lengths because of their strong correlation to scores (Vajjala, 2017). This has led to concern about the validity of the features used to establish the predictive accuracy of AES systems (Attali, 2007; Weigle, 2013). Reliance on construct-irrelevant features in ELL contexts also forfeits the opportunity to provide meaningful diagnostic feedback to test-takers or provide the second language acquisition (SLA) field with real insights (C.-F. E. Chen & Cheng, 2008). This thesis seeks to improve the validity and reliability of an AES system developed for ELL essays by employing a new set of features based on the acquisition order of English negation. Modest improvements were made to a baseline AES system's accuracy, showing the possibility and importance of engineering features relevant to the construct being assessed in ELL essays. In addition to these findings, a novel ordering of the sequence of English negation acquisition not previously described in SLA research emerged.

Degree

College and Department

Humanities; Linguistics and English Language

Rights

http://lib.byu.edu/about/copyright/