Using Observation Protocol Scores to Make Inferences About Change in Teacher Practices

Alzen, Jessica Lynn

Graduate Thesis Or Dissertation

Using Observation Protocol Scores to Make Inferences About Change in Teacher Practices Public Deposited

Analytics

Citations

Citeable URL: https://scholar.colorado.edu/concern/graduate_thesis_or_dissertations/vq27zn538

Abstract

Research on teacher learning and teacher change indicates that it is not unreasonable to expect teachers at all stages of their careers to change in their practices. However, measures of such change traditionally take the form of self-reflection and observation following teacher preparation programs (Grossman, Valencia, Evans, Thompson, Martin, & Place, 2000) or survey responses following professional development activities (Garet, Porter, Desimone, Birman, & Yoon, 2001). Even though observation protocols historically serve as the method for measuring teacher practices generally, they have yet to be used to understand change in teacher practices over time due to data limitations. Recent changes in teacher evaluation systems initiated more frequent and consistent teacher observations (Doherty & Stevens, 2015), so change in teacher practices as measured by observation protocols might soon be of greater interest to researchers and school leaders alike. Fortunately, the Measures of Effective Teaching (MET) project provides sufficient data for beginning to understand longitudinal changes in teacher practices. The two key contributions of this project are first, an application of hierarchical linear models to estimate growth over time in teacher observation scores and second, a careful investigation of the conditions that maximize the reliability of those growth estimates. The findings of this study suggest that teacher observation scores may change by about half of a standard deviation during a two-year time span for a few teachers, but most will show much more modest rates of growth. Further, the reliability of the growth parameter estimates can reach as high as 0.5, but the number and spacing of observation occasions as well as number of raters required to reach such levels of reliability may be too high for practical use in some districts. The HLM estimates in this study make an initial contribution to the research literature regarding the modeling of growth in observation scores over time. The reliability investigation provides practical information about observation system designs with the potential to yield maximally reliable estimates of growth. The former analysis gives context for future work regarding growth in observation scores while the latter informs decision-makers regarding the best choices in designing observation systems if longitudinal growth estimates are a target measure of interest.

Creator