Fundamentals of statistical signal processing: estimation theory
Fundamentals of statistical signal processing: estimation theory
Deriving traffic demands for operational IP networks: methodology and experience
IEEE/ACM Transactions on Networking (TON)
Traffic matrix estimation: existing techniques and new directions
Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications
Fast accurate computation of large-scale IP traffic matrices from link loads
SIGMETRICS '03 Proceedings of the 2003 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
An information-theoretic approach to traffic matrix estimation
Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications
Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
How to identify and estimate the largest traffic matrix elements in a dynamic environment
Proceedings of the joint international conference on Measurement and modeling of computer systems
Diagnosing network-wide traffic anomalies
Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications
Traffic matrix estimation on a large IP backbone: a comparison on real data
Proceedings of the 4th ACM SIGCOMM conference on Internet measurement
A distributed approach to measure IP traffic matrices
Proceedings of the 4th ACM SIGCOMM conference on Internet measurement
Data streaming algorithms for accurate and efficient measurement of traffic and flow matrices
SIGMETRICS '05 Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Traffic matrices: balancing measurements, inference and modeling
SIGMETRICS '05 Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
IMC '05 Proceedings of the 5th ACM SIGCOMM conference on Internet Measurement
OSPF monitoring: architecture, design and deployment experience
NSDI'04 Proceedings of the 1st conference on Symposium on Networked Systems Design and Implementation - Volume 1
Finding a needle in a haystack: pinpointing significant BGP routing changes in an IP network
NSDI'05 Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation - Volume 2
A data streaming algorithm for estimating entropies of od flows
Proceedings of the 7th ACM SIGCOMM conference on Internet measurement
Optimal sampling in state space models with applications to network monitoring
SIGMETRICS '08 Proceedings of the 2008 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
CSAMP: a system for network-wide flow monitoring
NSDI'08 Proceedings of the 5th USENIX Symposium on Networked Systems Design and Implementation
Uncovering Artifacts of Flow Measurement Tools
PAM '09 Proceedings of the 10th International Conference on Passive and Active Network Measurement
Packet doppler: network monitoring using packet shift detection
CoNEXT '08 Proceedings of the 2008 ACM CoNEXT Conference
Spatio-temporal compressive sensing and internet traffic matrices
Proceedings of the ACM SIGCOMM 2009 conference on Data communication
Deriving cramér-rao bounds and maximum likelihood estimators for traffic matrix inference
ACM SIGMETRICS Performance Evaluation Review
Survey of SNMP performance analysis studies
International Journal of Network Management
IM'09 Proceedings of the 11th IFIP/IEEE international conference on Symposium on Integrated Network Management
Ensemble routing for datacenter networks
Proceedings of the 6th ACM/IEEE Symposium on Architectures for Networking and Communications Systems
OpenTM: traffic matrix estimator for OpenFlow networks
PAM'10 Proceedings of the 11th international conference on Passive and active measurement
Network prefix-level traffic profiling: Characterizing, modeling, and evaluation
Computer Networks: The International Journal of Computer and Telecommunications Networking
Leveraging router programmability for traffic matrix computation
Proceedings of the Workshop on Programmable Routers for Extensible Services of Tomorrow
A case study of the accuracy of SNMP measurements
Journal of Electrical and Computer Engineering
Spatio-temporal compressive sensing and internet traffic matrices
IEEE/ACM Transactions on Networking (TON)
A compressive sensing-based reconstruction approach to network traffic
Computers and Electrical Engineering
Hi-index | 0.00 |
Estimation of traffic matrices, which provide critical input for network capacity planning and traffic engineering, has recently been recognized as an important research problem. Most of the previous approaches infer traffic matrix from either SNMP link loads or sampled NetFlow records. In this work, we design novel inference techniques that, by statistically correlating SNMP link loads and sampled NetFlow records, allow for much more accurate estimation of traffic matrices than obtainable from either information source alone, even when sampled NetFlow records are available at only a subset of ingress. Our techniques are practically important and useful since both SNMP and NetFlow are now widely supported by vendors and deployed in most of the operational IP networks. More importantly, this research leads us to a new insight that SNMP link loads and sampled NetFlow records can serve as "error correction codes" to each other. This insight helps us to solve a challenging open problem in traffic matrix estimation, "How to deal with dirty data (SNMP and NetFlow measurement errors due to hardware/software/transmission problems)?" We design techniques that, by comparing notes between the above two information sources, identify and remove dirty data, and therefore allow for accurate estimation of the traffic matrices with the cleaned dat.We conducted experiments on real measurement data obtained from a large tier-1 ISP backbone network. We show that, when full deployment of NetFlow is not available, our algorithm can improve estimation accuracy significantly even with a small fraction of NetFlow data. More importantly, we show that dirty data can contaminate a traffic matrix, and identifying and removing them can reduce errors in traffic matrix estimation by up to an order of magnitude. Routing changes is another a key factor that affects estimation accuracy. We show that using them as the a priori, the traffic matrices can be estimated much more accurately than those omitting the routing change. To the best of our knowledge, this work is the first to offer a comprehensive solution which fully takes advantage of using multiple readily available data sources. Our results provide valuable insights on the effectiveness of combining flow measurement and link load measurement.