A linear-time probabilistic counting algorithm for database applications
ACM Transactions on Database Systems (TODS)
Randomized algorithms
Space/time trade-offs in hash coding with allowable errors
Communications of the ACM
Introduction to Algorithms
New directions in traffic measurement and accounting
Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications
What's hot and what's not: tracking most frequent items dynamically
Proceedings of the twenty-second ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Bitmap algorithms for counting active flows on high speed links
Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
Sketch-based change detection: methods, evaluation, and applications
Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
Data streaming algorithms for efficient and accurate estimation of flow size distribution
Proceedings of the joint international conference on Measurement and modeling of computer systems
Online identification of hierarchical heavy hitters: algorithms, evaluation, and applications
Proceedings of the 4th ACM SIGCOMM conference on Internet measurement
Snort - Lightweight Intrusion Detection for Networks
LISA '99 Proceedings of the 13th USENIX conference on System administration
FlowScan: A Network Traffic Flow Reporting and Visualization Tool
LISA '00 Proceedings of the 14th USENIX conference on System administration
Data streaming algorithms for estimating entropy of network traffic
SIGMETRICS '06/Performance '06 Proceedings of the joint international conference on Measurement and modeling of computer systems
Joint data streaming and sampling techniques for detection of super sources and destinations
IMC '05 Proceedings of the 5th ACM SIGCOMM conference on Internet Measurement
A data streaming algorithm for estimating entropies of od flows
Proceedings of the 7th ACM SIGCOMM conference on Internet measurement
Reversible sketches: enabling monitoring and analysis over high-speed data streams
IEEE/ACM Transactions on Networking (TON)
A resource-minimalist flow size histogram estimator
Proceedings of the 8th ACM SIGCOMM conference on Internet measurement
A new data streaming method for locating hosts with large connection degree
GLOBECOM'09 Proceedings of the 28th IEEE conference on Global telecommunications
Computer Networks: The International Journal of Computer and Telecommunications Networking
A Data Streaming Method for Monitoring Host Connection Degrees of High-Speed Links
IEEE Transactions on Information Forensics and Security - Part 2
Hi-index | 0.00 |
It is difficult to accurately measure node connection degrees for a high speed network, since there is a massive amount of traffic to be processed. In this paper, we present a new virtual indexing method for estimating node connection degrees for high speed links. It is based on the virtual connection degree sketch (VCDS) where a compact sketch of network traffic is built by generating multiple virtual bitmaps for each network node. Each virtual bitmap consists of a fixed number of bits selected randomly from a shared bit array by a new method for recording the traffic flows of the corresponding node. The shared bit array is efficiently utilized by all nodes since every bit is shared by the virtual bitmaps of multiple nodes. To reduce the ''noise'' contaminated in a node's virtual bitmaps due to sharing, we propose a new method to generate the ''filtered'' bitmap used to estimate node connection degree. Furthermore, we apply VCDS to detect super nodes often associated with traffic anomalies. Since VCDS need a large amount of extra memory to store node addresses, we also propose a new data structure, the reversible virtual connection degree sketch, which identifies super node addresses analytically without the need of extra memory space but at a small increase in estimation error. Furthermore we combine the VCDS and RVCDS based methods with a uniform flow sampling technique to reduce memory complexities. Experiments are performed based on the actual network traffic and testing results show that the new methods are more memory efficient and more accurate than existing methods.