Improving Human-Classifier Interaction through Enhanced Highlighting Techniques

Kneusel, Ronald Thomas

Graduate Thesis Or Dissertation

Improving Human-Classifier Interaction through Enhanced Highlighting Techniques Public Deposited

Analytics

Download PDF

Citations

Citeable URL: https://scholar.colorado.edu/concern/graduate_thesis_or_dissertations/4m90dv839

Abstract

This dissertation is concerned with developing techniques to improve detection of target objects in digital imagery, e.g., satellite image analysis, airport baggage screening, and medical image diagnosis. Human experts are fairly good at these tasks, but expertise takes years to acquire and human performance is fallible. Computer systems trained through machine learning methods are promising, but in many difficult tasks computer systems have not yet reached the level of performance of human experts. This dissertation proposes an approach to human-computer cooperative analysis to obtain results that are better than either human or computer alone could achieve. The traditional route to improving human performance with results from automatic classifiers is to highlight images by drawing boxes around regions of an image that the computer system believes likely to contain a target object. Human experts typically do not like this form of assistance: it's often obvious to the expert that the highlighted region is relevant or irrelevant, and highlighting some regions often causes other regions to be overlooked. This dissertation proposes an alternative to the hard highlighting technique of drawing boxes around candidate targets. The alternative, soft highlighting, provides graded saliency cues based on the confidence level of a classifier. For example, with grey scale satellite imagery, soft highlighting might take the form of varying the saturation level of a particular hue. The dissertation describes a series of 8 experiments to evaluate the costs and benefits of soft highlighting versus hard highlighting versus a control condition of no highlighting. In Experiments 1-5, subjects search an array of handprinted digits for a given target digit identity. The elements of the array are highlighted according to the output of a classifier. The quality of the classifier was manipulated using a stochastic, oracle-based classifier that simulates classification to achieve a specified degree of discriminability between targets and nontargets. The experiments measured the time to locate targets in the array. Soft highlighting allows subjects to find targets faster than hard highlighting or the no-highlight control, even for weak classifiers, i.e., classifiers which had little disrcriminative power. The experiments found that highlighting affects search slopes (the time to process each element in the display), meaning that search becomes more effcient with highlighting. Not only was search more effcient, but fewer targets are missed with soft highlighting versus hard highlighting. Experiments 6-8 used actual satellite imagery. Subjects searched images for a particular target, a McDonald's restaurant. Highlights were obtained from a state-of-the-art convolutional neural net classifier which output a continuous confidence level. Experiment 6-8 also found that subjects could locate a target more quickly and with fewer misses with soft highlighting than with either hard highlighting or the no-highlight control. However, highlighting -- both soft and hard -- yielded more false alarms (nontarget locations identified as potential targets). We argue that while false alarms are a problem for novices who do not yet have the skill to verify the presence of a target, experts should not suffer from this same problem.

Creator