Date of Award
Doctor of Philosophy (PhD)
Derek C. Briggs
Lorrie A. Shepard
The results of educational assessments are often used to compare teachers, schools, districts, or states. In particular, relating variability in student achievement between states to differences in states’ policies and practices is, and has been, an area of interest and research. The National Assessment of Educational Progress (NAEP) is currently the only nationally administered assessment and as such, facilitates state to state comparisons. However, state level comparisons of student achievement are limited – NAEP’s subscores provide no unique information above and beyond the overall score.
Rather than use traditionally defined subscores, an alternative is to find subsets of items on which states do differ and use these items as the basis for defining state-level dimensionality, and thus subscores. Camilli et al. (2008) developed and implemented this type of approach using data from the 2000 NAEP. In this dissertation, I seek to replicate and extend Camilli et al.’s approach through the use of a more general model - an exploratory multilevel, multidimensional item response theory (IRT) model (or, equivalently, a multilevel full-information item factor analysis model; Bock, Gibbons, & Muraki, 1988; Cai, 2010) - than the one used by Camilli et al. To do so, I apply this model to data from the 2000 and 2009 NAEP fourth grade mathematics assessments.
In their analysis of the 2000 data, Camilli et al. found four state-level dimensions. Two of these dimensions replicated in my analysis – a decimals and fractions dimension and a geometric relationships dimension. These two dimensions differ from the NAEP subscores in terms of (a) mathematical content and (b) patterns of correlations with key state-level variables. These trends suggest that the dimensions offer unique insight into state-level policies and processes. However, these two dimensions do not appear in the 2009 analysis – at least not unequivocally. Instead, the 2009 dimensions did deal with some of the same mathematical concepts as the 2000 dimensions, but these concepts are divided up differently. Overall, this work provides a platform from which state-level subscores, used for NAEP’s operational reporting, could be developed.
Dadey, Nathan L., "Getting More out of the National Assessment of Educational Progress: Investigating Dimensionality at the State-Level" (2015). School of Education Graduate Theses & Dissertations. 61.