Abstract
The Performance Assessment and Evaluation System (PAES) is used by all major universities in the state of Utah to measure the effective teaching skills of preservice candidates as they progress through their teaching preparation program. The resulting ratings are used to make high-stakes decisions relating to course completion as well as recommendation for licensure. This study analyzes the factor structure and tests for measurement invariance of PAES ratings assigned to 663 elementary education candidates at Brigham Young University across two measurement occasions. The candidates were rated by 30 clinical faculty associates. This study also examines the degree to which differential rater effects impact the PAES ratings of these candidates. A bifactor model, with a general factor measuring effective teaching skills measured through observation, and a specific factor measuring effective teaching skills evaluated through conversation best fit the model. Evidence of measurement invariance was found between evaluations completed for Practicum 1 and Practicum 2 candidates. This study also found that differential rater effects impact the PAES ratings of individual candidates, indicating that a candidate's rating may depend on which rater completed the evaluation. Similar research studies should be conducted to analyze the quality of PAES ratings of teacher candidates in the various secondary education programs at BYU. In addition, since the PAES is used at other teacher preparation colleges and universities in Utah, similar research studies should be conducted to examine the quality of PAES ratings of teacher candidates at these other institutions.
Degree
PhD
College and Department
David O. McKay School of Education; Educational Inquiry, Measurement, and Evaluation
Rights
https://lib.byu.edu/about/copyright/
BYU ScholarsArchive Citation
Steadman, Anna Kay, "An Analysis of the Factor Structure and Measurement Invariance of the Performance Assessment and Evaluation System Ratings of Preservice Teachers" (2023). Theses and Dissertations. 9849.
https://scholarsarchive.byu.edu/etd/9849
Date Submitted
2023-04-14
Document Type
Dissertation
Handle
http://hdl.lib.byu.edu/1877/etd12687
Keywords
longitudinal measurement invariance, performance-based assessment, inter-rater reliability, teacher certification
Language
english