Probabilistic counting algorithms for data base applications
Journal of Computer and System Sciences
A linear-time probabilistic counting algorithm for database applications
ACM Transactions on Database Systems (TODS)
Proceedings of the ACM SIGCOMM '98 conference on Applications, technologies, architectures, and protocols for computer communication
Controlling high bandwidth aggregates in the network
ACM SIGCOMM Computer Communication Review
Practical automated detection of stealthy portscans
Journal of Computer Security
Congestion control for high bandwidth-delay product networks
Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications
New directions in traffic measurement and accounting
Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications
Properties and prediction of flow statistics from sampled packet streams
Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment
Snort - Lightweight Intrusion Detection for Networks
LISA '99 Proceedings of the 13th USENIX conference on System administration
FlowScan: A Network Traffic Flow Reporting and Visualization Tool
LISA '00 Proceedings of the 14th USENIX conference on System administration
Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
Data streaming algorithms for efficient and accurate estimation of flow size distribution
Proceedings of the joint international conference on Measurement and modeling of computer systems
Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications
A scalable distributed information management system
Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications
On scalable attack detection in the network
Proceedings of the 4th ACM SIGCOMM conference on Internet measurement
On the difficulty of scalably detecting network attacks
Proceedings of the 11th ACM conference on Computer and communications security
A data streaming algorithm for estimating subpopulation flow size distribution
SIGMETRICS '05 Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
A robust system for accurate real-time summaries of internet traffic
SIGMETRICS '05 Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Data streaming algorithms for accurate and efficient measurement of traffic and flow matrices
SIGMETRICS '05 Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Fast and accurate traffic matrix measurement using adaptive cardinality counting
Proceedings of the 2005 ACM SIGCOMM workshop on Mining network data
Fast payload-based flow estimation for traffic monitoring and network security
Proceedings of the 2005 ACM symposium on Architecture for networking and communications systems
What's new: finding significant differences in network data streams
IEEE/ACM Transactions on Networking (TON)
IEEE/ACM Transactions on Networking (TON)
Bitmap algorithms for counting active flows on high-speed links
IEEE/ACM Transactions on Networking (TON)
Achieving multipoint-to-multipoint fairness with RCNWA
Journal of Systems Architecture: the EUROMICRO Journal
On synopses for distinct-value estimation under multiset operations
Proceedings of the 2007 ACM SIGMOD international conference on Management of data
Joint data streaming and sampling techniques for detection of super sources and destinations
IMC '05 Proceedings of the 5th ACM SIGCOMM conference on Internet Measurement
OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
An algorithm for approximate counting using limited memory resources
Proceedings of the 2007 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Tracking port scanners on the IP backbone
Proceedings of the 2007 workshop on Large scale attack defense
Why go logarithmic if we can go linear?: Towards effective distinct counting of search traffic
EDBT '08 Proceedings of the 11th international conference on Extending database technology: Advances in database technology
Detector SherLOCK: Enhancing TRW with Bloom filters under memory and performance constraints
Computer Networks: The International Journal of Computer and Telecommunications Networking
Load shedding in network monitoring applications
ATC'07 2007 USENIX Annual Technical Conference on Proceedings of the USENIX Annual Technical Conference
EF-Greedy: A Novel Garbage Collection Policy for Flash Memory Based Embedded Systems
ICCS '07 Proceedings of the 7th international conference on Computational Science, Part IV: ICCS 2007
A programmable architecture for scalable and real-time network traffic measurements
Proceedings of the 4th ACM/IEEE Symposium on Architectures for Networking and Communications Systems
A Space-Efficient Fair Packet Sampling Algorithm
APNOMS '08 Proceedings of the 11th Asia-Pacific Symposium on Network Operations and Management: Challenges for Next Generation Network Operations and Service Management
Robust network monitoring in the presence of non-cooperative traffic queries
Computer Networks: The International Journal of Computer and Telecommunications Networking
Counting Flows over Sliding Windows in High Speed Networks
NETWORKING '09 Proceedings of the 8th International IFIP-TC 6 Networking Conference
Entropy based adaptive flow aggregation
IEEE/ACM Transactions on Networking (TON)
Every microsecond counts: tracking fine-grain latencies with a lossy difference aggregator
Proceedings of the ACM SIGCOMM 2009 conference on Data communication
How expensive is link utilization?
NET-COOP'07 Proceedings of the 1st EuroFGI international conference on Network control and optimization
On-line predictive load shedding for network monitoring
NETWORKING'07 Proceedings of the 6th international IFIP-TC6 conference on Ad Hoc and sensor networks, wireless networks, next generation internet
Trading link utilization for queueing delays: An adaptive approach
Computer Communications
Aggregate computation over data streams
APWeb'08 Proceedings of the 10th Asia-Pacific web conference on Progress in WWW research and development
A new data streaming method for locating hosts with large connection degree
GLOBECOM'09 Proceedings of the 28th IEEE conference on Global telecommunications
Finding heavy distinct hitters in data streams
Proceedings of the twenty-third annual ACM symposium on Parallelism in algorithms and architectures
Predictive resource management of multiple monitoring applications
IEEE/ACM Transactions on Networking (TON)
A new algorithm for long flows Statistics—MGCBF
ICCS'06 Proceedings of the 6th international conference on Computational Science - Volume Part IV
Entropy based flow aggregation
NETWORKING'06 Proceedings of the 5th international IFIP-TC6 conference on Networking Technologies, Services, and Protocols; Performance of Computer and Communication Networks; Mobile and Wireless Communications Systems
Virtual indexing based methods for estimating node connection degrees
Computer Networks: The International Journal of Computer and Telecommunications Networking
Efficient and reliable low-power backscatter networks
Proceedings of the ACM SIGCOMM 2012 conference on Applications, technologies, architectures, and protocols for computer communication
Router support for fine-grained latency measurements
IEEE/ACM Transactions on Networking (TON)
Efficient and reliable low-power backscatter networks
ACM SIGCOMM Computer Communication Review - Special october issue SIGCOMM '12
Software defined traffic measurement with OpenSketch
nsdi'13 Proceedings of the 10th USENIX conference on Networked Systems Design and Implementation
Line speed accurate superspreader identification using dynamic error compensation
Computer Communications
Hi-index | 0.00 |
This paper presents a family of bitmap algorithms that address the problem of counting the number of distinct header patterns (flows) seen on a high speed link. Such counting can be used to detect DoS attacks and port scans, and to solve measurement problems. Counting is especially hard when processing must be done within a packet arrival time (8 nsec at OC-768 speeds) and, hence, must require only a small number of accesses to limited, fast memory. A naive solution that maintains a hash table requires several Mbytes because the number of flows can be above a million. By contrast, our new probabilistic algorithms take very little memory and are fast. The reduction in memory is particularly important for applications that run multiple concurrent counting instances. For example, we replaced the port scan detection component of the popular intrusion detection system Snort with one of our new algorithms. This reduced memory usage on a ten minute trace from 50 Mbytes to 5.6 Mbytes while maintaining a 99.77% probability of alarming on a scan within 6 seconds of when the large-memory algorithm would. The best known prior algorithm (probabilistic counting) takes 4 times more memory on port scan detection and 8 times more on a measurement application. Fundamentally, this is because our algorithms can be customized to take advantage of special features of applications such as a large number of instances that have very small counts or prior knowledge of the likely range of the count.