Context-Aware Anomaly Detection and Analysis Using Spatial-Temporal Data

Liu, Qi

Graduate Thesis Or Dissertation

Context-Aware Anomaly Detection and Analysis Using Spatial-Temporal Data Public Deposited

Analytics

Citations

Citeable URL: https://scholar.colorado.edu/concern/graduate_thesis_or_dissertations/qn59q4164

Abstract

With the thriving of sensing and internet-of-things technologies, an increasing number of research communities and industries are stepping into the Era of Big Data. Following this technology trend, the amount and complexity of data generated by each domain are growing exponentially. The demand for automated monitoring, detecting and analyzing unusual events from those data are also increasing. These predictive analyses seek to identify and capture meaningful patterns in massive, highly heterogeneous data from various domains such as environmental sensing and cyber-physical systems. However, performing analysis such as anomaly detection faces a variety of challenges. For instance, the lack of prior knowledge regarding what is normal and what is abnormal, and the power consumption limitation for low-profile computing devices. These challenges constrain the flexibility of analysis methods. All these pose real problems to existing anomaly detection methods. Most existing techniques for anomaly detection only consider the content of the data source, i.e., the data itself directly gathered from sensing devices, not taking the context of the data into consideration. Therefore, anomalies under complicated settings are difficult to be identified. Hence, it is essential to design anomaly detection methods, especially the feature space design under a specific anomaly context. The context can be semantic, spatial, or temporal. This thesis studies the context-aware data analysis approaches using spatial-temporal data. A general principle to design a context-aware data analysis framework for spatial-temporal data is proposed and investigated in three different problems: contextual anomaly detection in remotely sensed imagery, hierarchical context-aware fault diagnosis in photovoltaic systems and energy-efficient wearable computing empowered by context-aware predictive analysis. Results include: (1) an automated contextual anomaly detection approach is proposed and implemented. The method constructs and utilizes spatial-temporal neighborhood context. Average precision and recall of 98.1\% and 95.7\% for contextual outlier detection are achieved. Also, meaningful and validated unusual events are detected from remotely sensed imagery. (2) A new hierarchical context-aware anomaly detection algorithm is proposed. With this algorithm, the fault detection accuracy of large-scale photovoltaic systems improves by 20\% (from 63\% to 83\%) for top-100 detected anomalies, compared with existing solutions. (3) By identifying and predicting the intra-signal context, the proposed sparse adaptive sensing algorithm achieves 97.7\% accuracy with 76.9\% to 99\% reduced energy consumption (83.6\% average reduction under real-world testing). These three studies demonstrate the utility of combining the spatial-temporal context in any future big data anomaly detection.

Creator