Date of Award

Spring 1-1-2015

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Education

First Advisor

Derek C. Briggs

Second Advisor

Gregory Camilli

Third Advisor

Lorrie A. Shepard

Fourth Advisor

Andrew Maul

Fifth Advisor

Michael Stallings

Abstract

The results of educational assessments are often used to compare teachers, schools, districts, or states. In particular, relating variability in student achievement between states to differences in states’ policies and practices is, and has been, an area of interest and research. The National Assessment of Educational Progress (NAEP) is currently the only nationally administered assessment and as such, facilitates state to state comparisons. However, state level comparisons of student achievement are limited – NAEP’s subscores provide no unique information above and beyond the overall score.

Rather than use traditionally defined subscores, an alternative is to find subsets of items on which states do differ and use these items as the basis for defining state-level dimensionality, and thus subscores. Camilli et al. (2008) developed and implemented this type of approach using data from the 2000 NAEP. In this dissertation, I seek to replicate and extend Camilli et al.’s approach through the use of a more general model - an exploratory multilevel, multidimensional item response theory (IRT) model (or, equivalently, a multilevel full-information item factor analysis model; Bock, Gibbons, & Muraki, 1988; Cai, 2010) - than the one used by Camilli et al. To do so, I apply this model to data from the 2000 and 2009 NAEP fourth grade mathematics assessments.

In their analysis of the 2000 data, Camilli et al. found four state-level dimensions. Two of these dimensions replicated in my analysis – a decimals and fractions dimension and a geometric relationships dimension. These two dimensions differ from the NAEP subscores in terms of (a) mathematical content and (b) patterns of correlations with key state-level variables. These trends suggest that the dimensions offer unique insight into state-level policies and processes. However, these two dimensions do not appear in the 2009 analysis – at least not unequivocally. Instead, the 2009 dimensions did deal with some of the same mathematical concepts as the 2000 dimensions, but these concepts are divided up differently. Overall, this work provides a platform from which state-level subscores, used for NAEP’s operational reporting, could be developed.

Share

COinS