Random sampling with a reservoir
ACM Transactions on Mathematical Software (TOMS)
Data networks
IEEE/ACM Transactions on Networking (TON)
An engineering approach to computer networking: ATM networks, the Internet, and the telephone network
Efficient policies for carrying Web traffic over flow-switched networks
IEEE/ACM Transactions on Networking (TON)
Using approximate majorization to characterize protocol fairness
Proceedings of the 2001 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
New directions in traffic measurement and accounting
Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications
Overcoming Limitations of Sampling for Aggregation Queries
Proceedings of the 17th International Conference on Data Engineering
Fairness in Routing and Load Balancing
FOCS '99 Proceedings of the 40th Annual Symposium on Foundations of Computer Science
On Efficient Max-Min Fair Routing Algorithms
ISCC '03 Proceedings of the Eighth IEEE International Symposium on Computers and Communications
Gigascope: a stream database for network applications
Proceedings of the 2003 ACM SIGMOD international conference on Management of data
Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
Optimal combination of sampled network measurements
IMC '05 Proceedings of the 5th ACM SIGCOMM conference on Internet Measurement
Lexicographic Maxmin Fairness for Data Collection in Wireless Sensor Networks
IEEE Transactions on Mobile Computing
Priority sampling for estimation of arbitrary subset sums
Journal of the ACM (JACM)
A unified framework for max-min and min-max fairness with applications
IEEE/ACM Transactions on Networking (TON)
Reformulating the monitor placement problem: optimal network-wide sampling
CoNEXT '06 Proceedings of the 2006 ACM CoNEXT conference
CSAMP: a system for network-wide flow monitoring
NSDI'08 Proceedings of the 5th USENIX Symposium on Networked Systems Design and Implementation
Max-Min Fair Scheduling in Input-Queued Switches
IEEE Transactions on Parallel and Distributed Systems
Fast monitoring of traffic subpopulations
Proceedings of the 8th ACM SIGCOMM conference on Internet measurement
A Space-Efficient Fair Packet Sampling Algorithm
APNOMS '08 Proceedings of the 11th Asia-Pacific Symposium on Network Operations and Management: Challenges for Next Generation Network Operations and Service Management
Stream sampling for variance-optimal estimation of subset sums
SODA '09 Proceedings of the twentieth Annual ACM-SIAM Symposium on Discrete Algorithms
On the variance of subset sum estimation
ESA'07 Proceedings of the 15th annual European conference on Algorithms
Flowroute: inferring forwarding table updates using passive flow-level measurements
IMC '10 Proceedings of the 10th ACM SIGCOMM conference on Internet measurement
Learn more, sample less: control of volume and variance in network measurement
IEEE Transactions on Information Theory
I know what your packet did last hop: using packet histories to troubleshoot networks
NSDI'14 Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation
Hi-index | 0.00 |
Sampling is crucial for controlling resource consumption by internet traffic flow measurements. Routers use Packet Sampled NetFlow, and completed flow records are sampled in the measurement infrastructure. Recent research, motivated by the need of service providers to accurately measure both small and large traffic subpopulations, has focused on distributing a packet sampling budget amongst subpopulations. But long timescales of hardware development and lower bandwidth costs motivate post-measurement analysis of complete flow records at collectors instead. Sampling in collector databases then manages data volumes, yielding general purpose summaries that are rapidly queried to trigger drill-down analysis on a time limited window of full data. These are sufficiently small to be archived. This paper addresses the problem of distributing a sampling budget over subpopulations of flow records. Estimation accuracy goals are met by fairly sharing the budget. We establish a correspondence between the type of accuracy goal, and the flavor of fair sharing used. A streaming Max-Min Fair Sampling algorithm fairly shares the sampling budget across subpopulations, with sampling as a mechanism to deallocate budget. This provides timely samples and is robust against uncertainties in configuration and demand. We illustrate using flow records from an access router of a large ISP, where rates over interface traffic subpopulations vary over several orders of magnitude. We detail an implementation whose computational cost is no worse than subpopulation-oblivious sampling.