A linear-time probabilistic counting algorithm for database applications
ACM Transactions on Database Systems (TODS)
Randomized algorithms
Efficient Hardware Hashing Functions for High Performance Computers
IEEE Transactions on Computers
Space/time trade-offs in hash coding with allowable errors
Communications of the ACM
Trajectory sampling for direct traffic observation
IEEE/ACM Transactions on Networking (TON)
An analysis of using reflectors for distributed denial-of-service attacks
ACM SIGCOMM Computer Communication Review
Flash crowds and denial of service attacks: characterization and implications for CDNs and web sites
Proceedings of the 11th international conference on World Wide Web
New directions in traffic measurement and accounting
Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications
A simple algorithm for finding frequent elements in streams and bags
ACM Transactions on Database Systems (TODS)
Estimating flow distributions from sampled flow statistics
Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications
Bitmap algorithms for counting active flows on high speed links
Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
Data streaming algorithms for efficient and accurate estimation of flow size distribution
Proceedings of the joint international conference on Measurement and modeling of computer systems
Online identification of hierarchical heavy hitters: algorithms, evaluation, and applications
Proceedings of the 4th ACM SIGCOMM conference on Internet measurement
Snort - Lightweight Intrusion Detection for Networks
LISA '99 Proceedings of the 13th USENIX conference on System administration
FlowScan: A Network Traffic Flow Reporting and Visualization Tool
LISA '00 Proceedings of the 14th USENIX conference on System administration
Data streaming algorithms for accurate and efficient measurement of traffic and flow matrices
SIGMETRICS '05 Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Probabilistic lossy counting: an efficient algorithm for finding heavy hitters
ACM SIGCOMM Computer Communication Review
Information Assurance: Dependability and Security in Networked Systems
Information Assurance: Dependability and Security in Networked Systems
Tracking port scanners on the IP backbone
Proceedings of the 2007 workshop on Large scale attack defense
Detector SherLOCK: Enhancing TRW with Bloom filters under memory and performance constraints
Computer Networks: The International Journal of Computer and Telecommunications Networking
Estimating Local Cardinalities in a Multidimensional Multiset
AIMS '07 Proceedings of the 1st international conference on Autonomous Infrastructure, Management and Security: Inter-Domain Management
HiFIND: A high-speed flow-level intrusion detection approach with DoS resiliency
Computer Networks: The International Journal of Computer and Telecommunications Networking
An online framework for catching top spreaders and scanners
Computer Networks: The International Journal of Computer and Telecommunications Networking
A new data streaming method for locating hosts with large connection degree
GLOBECOM'09 Proceedings of the 28th IEEE conference on Global telecommunications
Computer Networks: The International Journal of Computer and Telecommunications Networking
Finding heavy distinct hitters in data streams
Proceedings of the twenty-third annual ACM symposium on Parallelism in algorithms and architectures
Virtual indexing based methods for estimating node connection degrees
Computer Networks: The International Journal of Computer and Telecommunications Networking
Per-flow traffic measurement through randomized counter sharing
IEEE/ACM Transactions on Networking (TON)
Spreader classification based on optimal dynamic bit sharing
IEEE/ACM Transactions on Networking (TON)
Line speed accurate superspreader identification using dynamic error compensation
Computer Communications
| Hi-index | 0.00 |
Detecting the sources or destinations that have communicated with a large number of distinct destinations or sources during a small time interval is an important problem in network measurement and security. Previous detection approaches are not able to deliver the desired accuracy at high link speeds (10 to 40 Gbps). In this work, we propose two novel algorithms that provide accurate and efficient solutions to this problem. Their designs are based on the insight that sampling and data streaming are often suitable for capturing different and complementary regions of the information spectrum, and a close collaboration between them is an excellent way to recover the complete information. Our first solution builds on the standard hash-based flow sampling algorithm. Its main innovation is that the sampled traffic is further filtered by a data streaming module which allows for much higher sampling rate and hence much higher accuracy. Our second solution is more sophisticated but offers higher accuracy. It combines the power of data streaming in efficiently estimating quantities associated with a given identity, and the power of sampling in collecting a list of candidate identities. The performance of both solutions are evaluated using both mathematical analysis and trace-driven experiments on real-world Internet traffic.