Randomized algorithms
Summary cache: a scalable wide-area web cache sharing protocol
IEEE/ACM Transactions on Networking (TON)
Space/time trade-offs in hash coding with allowable errors
Communications of the ACM
Charging from sampled network usage
IMW '01 Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement
New directions in traffic measurement and accounting
Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications
Properties and prediction of flow statistics from sampled packet streams
Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment
Efficient implementation of a statistics counter architecture
SIGMETRICS '03 Proceedings of the 2003 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Stable distributions, pseudorandom generators, embeddings and data stream computation
FOCS '00 Proceedings of the 41st Annual Symposium on Foundations of Computer Science
Estimating flow distributions from sampled flow statistics
Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications
Bitmap algorithms for counting active flows on high speed links
Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
Sketch-based change detection: methods, evaluation, and applications
Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
Data streaming algorithms for efficient and accurate estimation of flow size distribution
Proceedings of the joint international conference on Measurement and modeling of computer systems
Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications
Data streaming algorithms for estimating entropy of network traffic
SIGMETRICS '06/Performance '06 Proceedings of the joint international conference on Measurement and modeling of computer systems
Design of a novel statistics counter architecture with optimal space and time efficiency
SIGMETRICS '06/Performance '06 Proceedings of the joint international conference on Measurement and modeling of computer systems
Sketching unaggregated data streams for subpopulation-size queries
Proceedings of the twenty-sixth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Algorithms and estimators for accurate summarization of internet traffic
Proceedings of the 7th ACM SIGCOMM conference on Internet measurement
Processing top k queries from samples
CoNEXT '06 Proceedings of the 2006 ACM CoNEXT conference
Processing top-k queries from samples
Computer Networks: The International Journal of Computer and Telecommunications Networking
Fast monitoring of traffic subpopulations
Proceedings of the 8th ACM SIGCOMM conference on Internet measurement
A resource-minimalist flow size histogram estimator
Proceedings of the 8th ACM SIGCOMM conference on Internet measurement
Every microsecond counts: tracking fine-grain latencies with a lossy difference aggregator
Proceedings of the ACM SIGCOMM 2009 conference on Data communication
Coordinated weighted sampling for estimating aggregates over multiple weight assignments
Proceedings of the VLDB Endowment
A new data streaming method for locating hosts with large connection degree
GLOBECOM'09 Proceedings of the 28th IEEE conference on Global telecommunications
Computer Networks: The International Journal of Computer and Telecommunications Networking
High-speed per-flow traffic measurement with probabilistic multiplicity counting
INFOCOM'10 Proceedings of the 29th conference on Information communications
Network prefix-level traffic profiling: Characterizing, modeling, and evaluation
Computer Networks: The International Journal of Computer and Telecommunications Networking
Router support for fine-grained latency measurements
IEEE/ACM Transactions on Networking (TON)
Modeling residual-geometric flow sampling
IEEE/ACM Transactions on Networking (TON)
| Hi-index | 0.00 |
Statistical information about the flow sizes in the traffic passing through a network link helps a network operator to characterize network resource usage, infer traffic demands, detect traffic anomalies, and improve network performance through traffic engineering. Previous work on estimating the flow size distribution for the complete population of flows has produced techniques that either make inferences from sampled network traffic, or use data streaming approaches. In this work, we identify and solve a more challenging problem of estimating the size distribution and other statistical information about arbitrary subpopulations of flows. Inferring subpopulation flow statistics is more challenging than the complete population counterpart, since subpopulations of interest are often specified a posteriori (i.e., after the data collection is done), making it impossible for the data collection module to "plan in advance".Our solution consists of a novel mechanism that combines data streaming with traditional packet sampling to provide highly accurate estimates of subpopulation flow statistics. The algorithm employs two data collection modules operating in parallel --- a NetFlow-like packet sampler and a streaming data structure made up of an array of counters. Combining the data collected by these two modules, our estimation algorithm uses a statistical estimation procedure that correlates and decodes the outputs (observations) from both data collection modules to obtain flow statistics for any arbitrary subpopulation. Evaluations of this algorithm on real-world Internet traffic traces demonstrate its high measurement accuracy.