Traffic matrix estimation: existing techniques and new directions
Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications
New directions in traffic measurement and accounting
Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications
Stable distributions, pseudorandom generators, embeddings and data stream computation
FOCS '00 Proceedings of the 41st Annual Symposium on Foundations of Computer Science
Low-rate TCP-targeted denial of service attacks: the shrew vs. the mice and elephants
Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications
How to identify and estimate the largest traffic matrix elements in a dynamic environment
Proceedings of the joint international conference on Measurement and modeling of computer systems
Data streaming algorithms for accurate and efficient measurement of traffic and flow matrices
SIGMETRICS '05 Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Profiling internet backbone traffic: behavior models and applications
Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communications
Mining anomalies using traffic feature distributions
Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communications
Entropy Based Worm and Anomaly Detection in Fast IP Networks
WETICE '05 Proceedings of the 14th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprise
Robust traffic matrix estimation with imperfect information: making use of multiple data sources
SIGMETRICS '06/Performance '06 Proceedings of the joint international conference on Measurement and modeling of computer systems
Data streaming algorithms for estimating entropy of network traffic
SIGMETRICS '06/Performance '06 Proceedings of the joint international conference on Measurement and modeling of computer systems
Stable distributions, pseudorandom generators, embeddings, and data stream computation
Journal of the ACM (JACM)
Impact of packet sampling on anomaly detection metrics
Proceedings of the 6th ACM SIGCOMM conference on Internet measurement
Estimating entropy over data streams
ESA'06 Proceedings of the 14th conference on Annual European Symposium - Volume 14
A near-optimal algorithm for computing the entropy of a stream
SODA '07 Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms
Approximate frequency counts over data streams
VLDB '02 Proceedings of the 28th international conference on Very Large Data Bases
Estimating entropy and entropy norm on data streams
STACS'06 Proceedings of the 23rd Annual conference on Theoretical Aspects of Computer Science
DRAM is plenty fast for wirespeed statistics counting
ACM SIGMETRICS Performance Evaluation Review
SODA '09 Proceedings of the twentieth Annual ACM-SIAM Symposium on Discrete Algorithms
Proceedings of the 40th Conference on Winter Simulation
Optimal sampling from sliding windows
Proceedings of the twenty-eighth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
UAI '09 Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence
An online framework for catching top spreaders and scanners
Computer Networks: The International Journal of Computer and Telecommunications Networking
A new data streaming method for locating hosts with large connection degree
GLOBECOM'09 Proceedings of the 28th IEEE conference on Global telecommunications
Design and performance analysis of a DRAM-based statistics counter array architecture
Proceedings of the 5th ACM/IEEE Symposium on Architectures for Networking and Communications Systems
Fast moment estimation in data streams in optimal space
Proceedings of the forty-third annual ACM symposium on Theory of computing
Towards a universal sketch for origin-destination network measurements
NPC'11 Proceedings of the 8th IFIP international conference on Network and parallel computing
Optimal sampling from sliding windows
Journal of Computer and System Sciences
Virtual indexing based methods for estimating node connection degrees
Computer Networks: The International Journal of Computer and Telecommunications Networking
DRAM-based statistics counter array architecture with performance guarantee
IEEE/ACM Transactions on Networking (TON)
| Hi-index | 0.00 |
Entropy has recently gained considerable significance as an important metric for network measurement. Previous research has shown its utility in clustering traffic and detecting traffic anomalies. While measuring the entropy of the traffic observed at a single point has already been studied, an interesting open problem is to measure the entropy of the traffic between every origin-destination pair. In this paper, we propose the first solution to this challenging problem. Our sketch builds upon and extends the Lp sketch of Indyk with significant additional innovations. We present calculations showing that our data streaming algorithm is feasible for high link speeds using commodity CPU/memory at a reasonable cost. Our algorithm is shown to be very accurate in practice via simulations, using traffic traces collected at a tier-1 ISP backbone link.