A linear-time probabilistic counting algorithm for database applications
ACM Transactions on Database Systems (TODS)
Efficient Hardware Hashing Functions for High Performance Computers
IEEE Transactions on Computers
Deriving traffic demands for operational IP networks: methodology and experience
Proceedings of the conference on Applications, Technologies, Architectures, and Protocols for Computer Communication
Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications
Trajectory sampling for direct traffic observation
IEEE/ACM Transactions on Networking (TON)
PC based precision timing without GPS
SIGMETRICS '02 Proceedings of the 2002 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Traffic matrix estimation: existing techniques and new directions
Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications
New directions in traffic measurement and accounting
Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications
Properties and prediction of flow statistics from sampled packet streams
Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment
Fast accurate computation of large-scale IP traffic matrices from link loads
SIGMETRICS '03 Proceedings of the 2003 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Efficient implementation of a statistics counter architecture
SIGMETRICS '03 Proceedings of the 2003 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
An information-theoretic approach to traffic matrix estimation
Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications
Estimating flow distributions from sampled flow statistics
Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications
Bitmap algorithms for counting active flows on high speed links
Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
How to identify and estimate the largest traffic matrix elements in a dynamic environment
Proceedings of the joint international conference on Measurement and modeling of computer systems
Data streaming algorithms for efficient and accurate estimation of flow size distribution
Proceedings of the joint international conference on Measurement and modeling of computer systems
Robust traffic matrix estimation with imperfect information: making use of multiple data sources
SIGMETRICS '06/Performance '06 Proceedings of the joint international conference on Measurement and modeling of computer systems
Design of a novel statistics counter architecture with optimal space and time efficiency
SIGMETRICS '06/Performance '06 Proceedings of the joint international conference on Measurement and modeling of computer systems
Fisher information of sampled packets: an application to flow size estimation
Proceedings of the 6th ACM SIGCOMM conference on Internet measurement
Joint data streaming and sampling techniques for detection of super sources and destinations
IMC '05 Proceedings of the 5th ACM SIGCOMM conference on Internet Measurement
A data streaming algorithm for estimating entropies of od flows
Proceedings of the 7th ACM SIGCOMM conference on Internet measurement
A simple and efficient estimation method for stream expression cardinalities
VLDB '07 Proceedings of the 33rd international conference on Very large data bases
BRICK: a novel exact active statistics counter architecture
Proceedings of the 4th ACM/IEEE Symposium on Architectures for Networking and Communications Systems
Packet doppler: network monitoring using packet shift detection
CoNEXT '08 Proceedings of the 2008 ACM CoNEXT Conference
IM'09 Proceedings of the 11th IFIP/IEEE international conference on Symposium on Integrated Network Management
Estimating flow distribution by using difference information of multiple packet samplings
ICOIN'09 Proceedings of the 23rd international conference on Information Networking
An online framework for catching top spreaders and scanners
Computer Networks: The International Journal of Computer and Telecommunications Networking
BRICK: a novel exact active statistics counter architecture
IEEE/ACM Transactions on Networking (TON)
Towards a universal sketch for origin-destination network measurements
NPC'11 Proceedings of the 8th IFIP international conference on Network and parallel computing
Hi-index | 0.00 |
The traffic volume between origin/destination (OD) pairs in a network, known as traffic matrix, is essential for efficient network provisioning and traffic engineering. Existing approaches of estimating the traffic matrix, based on statistical inference and/or packet sampling, usually cannot achieve very high estimation accuracy. In this work, we take a brand new approach in attacking this problem. We propose a novel data streaming algorithm that can process traffic stream at very high speed (e.g., 40 Gbps) and produce traffic digests that are orders of magnitude smaller than the traffic stream. By correlating the digests collected at any OD pair using Bayesian statistics, the volume of traffic flowing between the OD pair can be accurately determined. We also establish principles and techniques for optimally combining this streaming method with sampling, when sampling is necessary due to stringent resource constraints. In addition, we propose another data streaming algorithm that estimates flow matrix, a finer-grained characterization than traffic matrix. Flow matrix is concerned with not only the total traffic between an OD pair (traffic matrix), but also how it splits into flows of various sizes. Through rigorous theoretical analysis and extensive synthetic experiments on real Internet traffic, we demonstrate that these two algorithms can produce very accurate estimation of traffic matrix and flow matrix respectively.