It has been proposed that intelligent autonomous systems that dynamically estimate operator cognitive states can adapt their behavior to better aid human operators and enhance team performance. This capability is crucial for astronaut support during deep space missions, where communication delays with ground control limit timely assistance. Previous research on cognitive state estimation has generally focused on single states, observable factors (e.g., task load), and the use of multiple sensors simultaneously.

While our laboratory demonstrated an adaptive autonomous system that changed modes based on real-time estimates of trust, workload, and situation awareness, it exhibited limited accuracy, potentially due to being trained on data from a non-adaptive system. Subsequent research performed a study that compared the predictive performance of multiple models with various feature availability variations, but did not account for the effects of mode changes. To address this, we developed multiple models trained on data from subjects interacting with both a non-adaptive and an adaptive autonomous system. First, we trained models on the group that interacted with the adaptive autonomous system, incorporating users' background information, real-time sensor data, human-system interactions, and novel features related to adaptation frequency and consistency of the autonomous system mode. Next, we trained and tested models built with exhaustive search followed by stepwise regression under optimizing for the Bayesian Information Criterion (BIC) using datasets from the adaptive group (N=10), the non-adaptive group (N=14), and a combined dataset (N=24). Finally, we explored three modeling methods with different feature down-selection approaches. Our results indicate that the combined dataset model focusing solely on human-system interaction features trained with exhaustive search followed by stepwise regression under BIC provided the most robust and best-performing prediction of operator trust, workload, and situation awareness. For example, the mean MAE (Mean Absolute Error) across 100 Monte Carlo Cross Validations for trust was 8.75, which was much smaller than the mean MAE by always predicting the median of the scale (as a baseline), which was 18.19. Similar results were found for models predicting workload and situation awareness.

This work enhances the state of the art in terms of the potential of adding new features that capture the interaction with an adaptive autonomous system, necessary datasets for training to have a more robust model when applied to unseen subjects, and approaches for building models to predict operator cognitive states for use in adaptive autonomous systems.