Date of Award

Spring 1-1-2015

Document Type


Degree Name

Master of Science (MS)


Computer Science

First Advisor

Elizabeth R. Jessup

Second Advisor

Allison H. Baker

Third Advisor

Dorit M. Hammerling

Fourth Advisor

Thomas Hauser


Complex, modular codes such as climate simulations are in a constant state of development, requiring frequent verification. Historically, this process has been computationally expensive and dependent on the interpretation of a climate scientist. In this research, we utilize the recently developed Community Earth System Model (CESM) Ensemble Consistency Test (CESM-ECT) to provide an objective measure of consistency for new CESM simulation runs. This tool relies on the creation of an ensemble of simulations that represent the same climate system, and whose statistical distribution embodies the innate variability of the climate model. A CESM ensemble is created by perturbations to the initial temperature; in this work we investigate whether such perturbations are adequate for characterizing the variability of a consistent climate. To this end we develop a systematic method for introducing minimal changes to the Community Atmospheric Model (CAM) component of CESM which are not expected to modify the climate, and measure the resultant numerical differences and their effects on the output ensemble with CESM-ECT. These minimal changes include mathematically identical reformulations (e.g. order of operation alteration and exchanging division with multiplication by inverse), compiler groups and versions, levels of compiler optimization, and substitution of the random number generator. Modifications to CAM modules are selected to be minimal in the sense that 1: the number of lines of modified code is minimized (i.e. limited to one distinct code block or math kernel), and 2: the alteration should not be climate-changing. These types of changes should translate to a CESM-ECT pass, and we examined the CESM-ECT ensemble in that context. In fact, the primary objective of this research was to ensure that the constituents of the CESM-ECT ensemble provide sufficient variability to represent a consistent climate, in effect verifying the CESM-ECT ensemble composition.

In this thesis, we demonstrate that the current CESM-ECT ensemble does not contain adequate variability to capture such minimal changes, and we suggest means of improvement to reach the desired false positive rate.