Date of Award

Spring 1-1-2012

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Education

First Advisor

Guillermo Solano-Flores

Second Advisor

Kathy Escamilla

Third Advisor

Janette Klingner

Fourth Advisor

Lucinda Soltero-Gonzalez

Fifth Advisor

Maria Thomas-Ruzic

Abstract

This dissertation addresses the complexity of test illustrations design across cultures. More specifically, it examines how the characteristics of illustrations used in science test items vary across content areas, assessment programs, and cultural origins. It compares a total of 416 Grade 8 illustrated items from the areas of earth science, life science, and physics used in an international assessment, four U.S. state assessments, and China's middle school exit examinations. It also examines the relationship of student performance and item illustrations' formal properties.

Results from both statistical analyses and analyses of exemplar illustrations show that although illustrations across cultures tended to have similar complexities, the sets of features that make them complex can be different. Illustrations from the Chinese tests provide more frequently than illustrations from the American tests visual information connecting the content of the item with the country's national, cultural, and historical contexts. Results also show that the contextual features in test illustrations tend to be positively related to student performance on the items.

This study is among the first approaches to analyzing the use of illustrations in the context of testing across cultures. Its results speak to the need for systematically examining the design and effectiveness of illustrations in science tests. Identifying cultural differences in the features of item illustrations can inform the process of item development and contribute to ensuring more valid assessment of culturally diverse populations in both the U.S. and the context of international test comparisons.