Keywords

Speaking Assessment, ASR, Elicited Imitation, Sentence Repetition

Abstract

This study examined the use of automatic speech recognition (ASR) scored elicited oral response (EOR) tests to assess the speaking ability of English language learners. It also examined the relationship between ASR-scored EOR and other language proficiency measures and the ability of the ASR to rate speakers without bias to gender or native language. To that end, 179 subjects were given an ASR-scored EOR test with 60 items, followed by an oral proficiency interview (OPI) type assessment and a battery of other language tests. Findings suggest that ASR-scored EOR results could be used alone to predict speaking ability in specific situations and for limited purposes such as initial placement of students in language training situations. However, if more certainty is required, adding a listening component would improve the assessment. Analysis of the study results also suggests that while there were some differences in amount of variance explained in speaking scores based on gender and native language, there was no significant negative effect that would preclude the use of ASR-scoring. While EOR is not an authentic performance assessment of the speaking ability, it does correlate well with other assessments of this construct and has good content validity. The use of an ASR-scored EOR test seems to provide a practical estimate of speaking proficiency that could be used for initial placement of students in situations where assessments of speaking for the purpose of placement are not currently being used due to the cost of adminstering OPI type assessments.

Original Publication Citation

Cox, T. & Davies, R. (2012) Using automatic speech recognition technology with elicited oral response testing. CALICO Journal, 29(4), 601-618