Establishing Reliable Procedures for Rating ELL Students’ Reading Comprehension Using Oral Retellings

Keywords

reading comprehension, second language learners, oral retellings

Abstract

Oral retellings are strongly recommended as a way to measure reading comprehension for second language learners (Bernhardt, 1985, 1990, 1991). However, the reliability of such ratings is a matter of concern for a variety of reasons (Aiken, 1996; Cooper, 1981; Saal, Downey, & Lahey, 1980). The purpose of this study was to establish reliable rating procedures to assess the reading comprehension of college‐age second language learners using oral retellings. A three‐facet (passage, rater, and rating occasion) generalizability study was conducted using oral retellings of expository passages. Retellings of three passages provided by 24 ELL students were rated by two trained raters on two occasions. The largest sources of error variance identified were the passages and the student‐by‐passage interaction. Results showed that the most important step that could be taken to reduce measurement error and increase generalizability would be to use at least four and preferably six passages to assess ELL students’ reading comprehension. The gain in generalizability from using additional raters or rating occasions was minimal.

Original Publication Citation

Sudweeks, R., Glissmeyer, C., Morrison, T., Wilcox, B. and Tanner, M. (2004). Establishing reliable procedures for rating ELL students’ reading comprehension using oral retellings. Reading Research and Instruction, 43(2), 65-86.

Document Type

Peer-Reviewed Article

Publication Date

2004

Permanent URL

http://hdl.lib.byu.edu/1877/8690

Publisher

Taylor and Francis Group

Language

English

College

Humanities

Department

Linguistics

University Standing at Time of Publication

Assistant Professor

Link to Full Text

Share

COinS