Fit a compact spread estimator in small high-speed memory

Authors:
MyungKeun Yoon;Tao Li;Shigang Chen;Jih-Kwon Peir
Affiliations:
Department of Computer Engineering, Kookmin University, Seoul, Korea;Department of Computer and Information Science and Engineering, University of Florida, Gainesville, FL;Department of Computer and Information Science and Engineering, University of Florida, Gainesville, FL;Department of Computer and Information Science and Engineering, University of Florida, Gainesville, FL
Venue:
IEEE/ACM Transactions on Networking (TON)
Year:
2011

Citing 12
Cited 1

A linear-time probabilistic counting algorithm for database applications

ACM Transactions on Database Systems (TODS)
Practical automated detection of stealthy portscans

Journal of Computer Security
New directions in traffic measurement and accounting

Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications
Sketch-based change detection: methods, evaluation, and applications

Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
Data streaming algorithms for efficient and accurate estimation of flow size distribution

Proceedings of the joint international conference on Measurement and modeling of computer systems
Online identification of hierarchical heavy hitters: algorithms, evaluation, and applications

Proceedings of the 4th ACM SIGCOMM conference on Internet measurement
Snort - Lightweight Intrusion Detection for Networks

LISA '99 Proceedings of the 13th USENIX conference on System administration
FlowScan: A Network Traffic Flow Reporting and Visualization Tool

LISA '00 Proceedings of the 14th USENIX conference on System administration
Bitmap algorithms for counting active flows on high-speed links

IEEE/ACM Transactions on Networking (TON)
Wire speed packet classification without tcams: a few more registers (and a bit of logic) are enough

Proceedings of the 2007 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Topological transformation approaches to optimizing TCAM-based packet classification systems

Proceedings of the eleventh international joint conference on Measurement and modeling of computer systems
Detection of Super Sources and Destinations in High-Speed Networks: Algorithms, Analysis and Evaluation

IEEE Journal on Selected Areas in Communications

A grand spread estimator using a graphics processing unit

Journal of Parallel and Distributed Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

The spread of a source host is the number of distinct destinations that it has sent packets to during a measurement period. A spread estimator is a software/hardware module on a router that inspects the arrival packets and estimates the spread of each source. It has important applications in detecting port scans and distributed denial-of-service (DDoS) attacks, measuring the infection rate of a worm, assisting resource allocation in a server farm, determining popular Web contents for caching, to name a few. The main technical challenge is to fit a spread estimator in a fast but small memory (such as SRAM) in order to operate it at the line speed in a high-speed network. In this paper, we design a new spread estimator that delivers good performance in tight memory space where all existing estimators no longer work. The new estimator not only achieves space compactness, but operates more efficiently than the existing ones. Its accuracy and efficiency come from a new method for data storage, called virtual vectors, which allow us to measure and remove the errors in spread estimation. We also propose several ways to enhance the range of spread values that the estimator can measure. We perform extensive experiments on real Internet traces to verify the effectiveness of the new estimator.