Randomized algorithms
New directions in traffic measurement and accounting
Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications
Efficient implementation of a statistics counter architecture
SIGMETRICS '03 Proceedings of the 2003 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Proceedings of the 2003 ACM SIGMOD international conference on Management of data
Sketch-based change detection: methods, evaluation, and applications
Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
Data streaming algorithms for efficient and accurate estimation of flow size distribution
Proceedings of the joint international conference on Measurement and modeling of computer systems
Online identification of hierarchical heavy hitters: algorithms, evaluation, and applications
Proceedings of the 4th ACM SIGCOMM conference on Internet measurement
A data streaming algorithm for estimating subpopulation flow size distribution
SIGMETRICS '05 Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Data streaming algorithms for accurate and efficient measurement of traffic and flow matrices
SIGMETRICS '05 Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
An improved data stream summary: the count-min sketch and its applications
Journal of Algorithms
An algorithm for approximate counting using limited memory resources
Proceedings of the 2007 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Counter braids: a novel counter architecture for per-flow measurement
SIGMETRICS '08 Proceedings of the 2008 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
CSAMP: a system for network-wide flow monitoring
NSDI'08 Proceedings of the 5th USENIX Symposium on Networked Systems Design and Implementation
DRAM is plenty fast for wirespeed statistics counting
ACM SIGMETRICS Performance Evaluation Review
BRICK: a novel exact active statistics counter architecture
Proceedings of the 4th ACM/IEEE Symposium on Architectures for Networking and Communications Systems
Small synopses for group-by query verification on outsourced data streams
ACM Transactions on Database Systems (TODS)
A locally encodable and decodable compressed data structure
Allerton'09 Proceedings of the 47th annual Allerton conference on Communication, control, and computing
Computer Networks: The International Journal of Computer and Telecommunications Networking
High-speed per-flow traffic measurement with probabilistic multiplicity counting
INFOCOM'10 Proceedings of the 29th conference on Information communications
Revisiting the case for a minimalist approach for network flow monitoring
IMC '10 Proceedings of the 10th ACM SIGCOMM conference on Internet measurement
Design and performance analysis of a DRAM-based statistics counter array architecture
Proceedings of the 5th ACM/IEEE Symposium on Architectures for Networking and Communications Systems
BotGrep: finding P2P bots with structured graph analysis
USENIX Security'10 Proceedings of the 19th USENIX conference on Security
Better by a HAIR: hardware-amenable Internet routing
Computer Networks: The International Journal of Computer and Telecommunications Networking
Online measurement of large traffic aggregates on commodity switches
Hot-ICE'11 Proceedings of the 11th USENIX conference on Hot topics in management of internet, cloud, and enterprise networks and services
BRICK: a novel exact active statistics counter architecture
IEEE/ACM Transactions on Networking (TON)
Fast dynamic multiple-set membership testing using combinatorial bloom filters
IEEE/ACM Transactions on Networking (TON)
DRAM-based statistics counter array architecture with performance guarantee
IEEE/ACM Transactions on Networking (TON)
Per-flow traffic measurement through randomized counter sharing
IEEE/ACM Transactions on Networking (TON)
Software defined traffic measurement with OpenSketch
nsdi'13 Proceedings of the 10th USENIX conference on Networked Systems Design and Implementation
Spreader classification based on optimal dynamic bit sharing
IEEE/ACM Transactions on Networking (TON)
Modeling residual-geometric flow sampling
IEEE/ACM Transactions on Networking (TON)
Hi-index | 0.00 |
The problem of how to efficiently maintain a large number (say millions) of statistics counters that need to be incremented at very high speed has received considerable research attention recently. This problem arises in a variety of router management algorithms and data streaming algorithms, where a large array of counters is used to track various network statistics and to implement various counting sketches respectively. While fitting these counters entirely in SRAM meets the access speed requirement, a large amount of SRAM may be needed with a typical counter size of 32 or 64 bits, and hence the high cost. Solutions proposed in recent works have used hybrid architectures where small counters in SRAM are incremented at high speed, and occasionally written back ("flushed") to larger counters in DRAM. Previous solutions have used complex schedulers with tree-like or heap data structures to pick which counters in SRAM are about to overflow, and flush them to the corresponding DRAM counters.In this work, we present a novel hybrid SRAM/DRAM counter architecture that consumes much less SRAM and has a much simpler design of the scheduler than previous approaches. We show, in fact, that our design is optimal in the sense that for a given speed difference between SRAM and DRAM, our design uses the theoretically minimum number of bits per counter in SRAM. Our design uses a small write-back buffer (in SRAM) that stores indices of the overflowed counters (to be flushed to DRAM) and an extremely simple randomized algorithm to statistically guarantee that SRAM counters do not overflow in bursts large enough to fill up the write-back buffer even in the worst case. The statistical guarantee of the algorithm is proven using a combination of worst case analysis for characterizing the worst case counter increment sequence and a new tail bound theorem for bounding the probability of filling up the write-back buffer. Experiments with real Internet traffic traces show that the buffer size required in practice is significantly smaller than needed in the worst case.