A scalable, non-parametric anomaly detection framework for Hadoop

Authors:
Li Yu;Zhiling Lan
Affiliations:
Illinois Institute of Technology, Chicago, IL;Illinois Institute of Technology, Chicago, IL
Venue:
Proceedings of the 2013 ACM Cloud and Autonomic Computing Conference
Year:
2013

Citing 4
Cited 0

Nonlinear component analysis as a kernel eigenvalue problem

Neural Computation
Mean Shift Based Clustering in High Dimensions: A Texture Classification Example

ICCV '03 Proceedings of the Ninth IEEE International Conference on Computer Vision - Volume 2
MapReduce: simplified data processing on large clusters

OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Large scale debugging of parallel tasks with AutomaDeD

Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we present a scalable and practical problem diagnosis framework for Hadoop environments. Our design features a decentralized approach based on hierarchical grouping and a novel non-parametric diagnostic mechanism. We evaluate our framework under various Hadoop workloads. The experimental results show that our design outperforms traditional methods significantly in the context of complex anomaly patterns and high anomaly probability.