Date of Award

Spring 1-1-2012

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Education

First Advisor

Kathy Escamilla

Second Advisor

Bhuvana Narasimhan

Third Advisor

Richard Shavelson

Fourth Advisor

Guillermo Solano-Flores

Fifth Advisor

Lucinda Soltero-González

Abstract

This study addresses the complex process of translation and adaptation of two Collegiate Learning Assessment (CLA) performance tasks (PTs), originally developed in English for American students, into the languages and cultures of five participating countries. Focusing on confirming evidence bits (CEBs), disconfirming evidence bits (DEBs), and no evidence bits (NEBs), I captured information regarding the factors that are critical to completing the translation and adaptation process as intended. Fidelity of Implementation (FOI) was examined by tasks to be completed and criteria to be met. Using a FOI framework created for this study, I examined information from 100 documents, emails, interviews, meetings minutes, and surveys and coded results according to evidence type. The framework includes 18 tasks that countries were to complete and 11 criteria with which countries were to comply. More specifically, I analyzed results from Spearman correlations for CEBs between countries and examined the relationship between CEBs and DEBs across tasks and criteria. In addition, I computed a fidelity of implementation (F) coefficient based on CEBs and DEBs by task and criterion as well as on aggregate for each category. Qualitative analysis provided contextual information explaining the F coefficients obtained, the Spearman correlation results, and the CEB to DEB ratios computed.

Results from this study point at specific aspects of countries’ political, educational, linguistic, and cultural contexts that shape FOI during test translation and adaptation. Based on these findings, I share three recommendations for future test translation projects.

Share

COinS