The space complexity of approximating the frequency moments
STOC '96 Proceedings of the twenty-eighth annual ACM symposium on Theory of computing
Summary cache: a scalable wide-area web cache sharing protocol
IEEE/ACM Transactions on Networking (TON)
Space/time trade-offs in hash coding with allowable errors
Communications of the ACM
New directions in traffic measurement and accounting
Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications
Finding Frequent Items in Data Streams
ICALP '02 Proceedings of the 29th International Colloquium on Automata, Languages and Programming
Frequency Estimation of Internet Packet Streams with Limited Space
ESA '02 Proceedings of the 10th Annual European Symposium on Algorithms
A simple algorithm for finding frequent elements in streams and bags
ACM Transactions on Database Systems (TODS)
Proceedings of the 2003 ACM SIGMOD international conference on Management of data
Gigascope: a stream database for network applications
Proceedings of the 2003 ACM SIGMOD international conference on Management of data
Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
Identifying elephant flows through periodically sampled packets
Proceedings of the 4th ACM SIGCOMM conference on Internet measurement
CAPTRA: coordinated packet traceback
Proceedings of the 5th international conference on Information processing in sensor networks
IEEE/ACM Transactions on Networking (TON)
Finding global icebergs over distributed data sets
Proceedings of the twenty-fifth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Approximate encoding for direct access and query processing over compressed bitmaps
VLDB '06 Proceedings of the 32nd international conference on Very large data bases
Reducing energy of virtual cache synonym lookup using bloom filters
CASES '06 Proceedings of the 2006 international conference on Compilers, architecture and synthesis for embedded systems
Proceedings of the 6th ACM SIGCOMM conference on Internet measurement
A network mitigation system against distributed denial of service: a linux-based prototype
IMSA'07 IASTED European Conference on Proceedings of the IASTED European Conference: internet and multimedia systems and applications
EF-Greedy: A Novel Garbage Collection Policy for Flash Memory Based Embedded Systems
ICCS '07 Proceedings of the 7th international conference on Computational Science, Part IV: ICCS 2007
A network mitigation system against distributed denial of service: a Linux-based prototype
EurolMSA '07 Proceedings of the Third IASTED European Conference on Internet and Multimedia Systems and Applications
Fast track article: Connectivity monitoring in wireless sensor networks
Pervasive and Mobile Computing
Computer Networks: The International Journal of Computer and Telecommunications Networking
Enhancing counting bloom filters through Huffman-coded multilayer structures
IEEE/ACM Transactions on Networking (TON)
A Generalized Bloom Filter to Secure Distributed Network Applications
Computer Networks: The International Journal of Computer and Telecommunications Networking
A multi-attribute data structure with parallel bloom filters for network services
HiPC'06 Proceedings of the 13th international conference on High Performance Computing
A new algorithm for long flows Statistics—MGCBF
ICCS'06 Proceedings of the 6th international conference on Computational Science - Volume Part IV
An algorithm for estimation of flow length distributions using heavy-tailed feature
ICCS'06 Proceedings of the 6th international conference on Computational Science - Volume Part IV
String matching on the internet
CAAN'04 Proceedings of the First international conference on Combinatorial and Algorithmic Aspects of Networking
Space-efficient structures for detecting port scans
DEXA'07 Proceedings of the 18th international conference on Database and Expert Systems Applications
Hi-index | 0.01 |
Per-flow traffic measurement is critical for usage accounting, traffic engineering, and anomaly detection. Previous methodologies are either based on random sampling (e.g., Cisco's NetFlow), which is inaccurate, or only account for the "elephants". Our paper introduces a novel technique for measuring per-flow traffic approximately, for all flows regardless of their sizes, at very high-speed (say, OC192+). The core of this technique is a novel data structure called Space Code Bloom Filter (SCBF). A SCBF is an approximate representation of a multiset; each element in this multiset is a traffic flow and its multiplicity is the number of packets in the flow. SCBF employs a Maximum Likelihood Estimation (MLE) method to measure the multiplicity of an element in the multiset. Through parameter tuning, SCBF allows for graceful tradeoff between measurement accuracy and computational and storage complexity. SCBF also contributes to the foundation of data streaming by introducing a new paradigm called blind streaming. We evaluated the performance of SCBF on packet traces gathered from a tier-1 ISP backbone and through mathematical analysis. Our preliminary results demonstrate that SCBF achieves reasonable mea-surement accuracy with very low storage and computational complexity.