Structural analysis of network traffic matrix via relaxed principal component pursuit

Authors:
Zhe Wang;Kai Hu;Ke Xu;Baolin Yin;Xiaowen Dong
Affiliations:
School of Computer Science and Engineering, Beihang University, Beijing 100191, China and State Key Laboratory of Software Development Environment, Beihang University, Beijing 100191, China;School of Computer Science and Engineering, Beihang University, Beijing 100191, China;School of Computer Science and Engineering, Beihang University, Beijing 100191, China and State Key Laboratory of Software Development Environment, Beihang University, Beijing 100191, China;School of Computer Science and Engineering, Beihang University, Beijing 100191, China and State Key Laboratory of Software Development Environment, Beihang University, Beijing 100191, China;Signal Processing Laboratories (LTS4/LTS2), Ecole Polytechnique Fédérale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland
Venue:
Computer Networks: The International Journal of Computer and Telecommunications Networking
Year:
2012

Citing 16
Cited 1

On the early history of the singular value decomposition

SIAM Review
Atomic Decomposition by Basis Pursuit

SIAM Review
Structural analysis of network traffic flows

Proceedings of the joint international conference on Measurement and modeling of computer systems
Diagnosing network-wide traffic anomalies

Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications
Traffic matrices: balancing measurements, inference and modeling

SIGMETRICS '05 Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Providing public intradomain traffic matrices to the research community

ACM SIGCOMM Computer Communication Review
Network anomography

IMC '05 Proceedings of the 5th ACM SIGCOMM conference on Internet Measurement
Combining filtering and statistical methods for anomaly detection

IMC '05 Proceedings of the 5th ACM SIGCOMM conference on Internet Measurement
Sensitivity of PCA for traffic anomaly detection

Proceedings of the 2007 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Uncovering Artifacts of Flow Measurement Tools

PAM '09 Proceedings of the 10th International Conference on Passive and Active Network Measurement
ANTIDOTE: understanding and defending against poisoning of anomaly detectors

Proceedings of the 9th ACM SIGCOMM conference on Internet measurement conference
Exact Matrix Completion via Convex Optimization

Foundations of Computational Mathematics
Decomposing background topics from keywords by principal component pursuit

CIKM '10 Proceedings of the 19th ACM international conference on Information and knowledge management
A Singular Value Thresholding Algorithm for Matrix Completion

SIAM Journal on Optimization
Compressed sensing

IEEE Transactions on Information Theory
De-noising by soft-thresholding

IEEE Transactions on Information Theory

A power laws-based reconstruction approach to end-to-end network traffic

Journal of Network and Computer Applications

Quantified Score

Hi-index	0.00

Visualization

Abstract

The network traffic matrix is widely used in network operation and management. It is therefore of crucial importance to analyze the components and the structure of the network traffic matrix, for which several mathematical approaches such as Principal Component Analysis (PCA) were proposed. In this paper, we first argue that PCA performs poorly for analyzing traffic matrix that is polluted by large volume anomalies, and then propose a new decomposition model for the network traffic matrix. According to this model, we carry out the structural analysis by decomposing the network traffic matrix into three sub-matrices, namely, the deterministic traffic, the anomaly traffic and the noise traffic matrix, which is similar to the Robust Principal Component Analysis (RPCA) problem previously studied in [13]. Based on the Relaxed Principal Component Pursuit (Relaxed PCP) method and the Accelerated Proximal Gradient (APG) algorithm, we present an iterative approach for decomposing a traffic matrix, and demonstrate its efficiency and flexibility by experimental results. Finally, we further discuss several features of the deterministic and noise traffic. Our study develops a novel method for the problem of structural analysis of the traffic matrix, which is robust against pollution of large volume anomalies.