Tabulation based 4-universal hashing with applications to second moment estimation

Authors:
Mikkel Thorup;Yin Zhang
Affiliations:
AT&T Labs---Research, Shannon Laboratory, Florham Park, NJ;AT&T Labs---Research, Shannon Laboratory, Florham Park, NJ
Venue:
SODA '04 Proceedings of the fifteenth annual ACM-SIAM symposium on Discrete algorithms
Year:
2004

Citing 14
Cited 48

The computational complexity of universal hashing

Theoretical Computer Science - Special issue on structure in complexity theory
Effective erasure codes for reliable computer communication protocols

ACM SIGCOMM Computer Communication Review
A reliable randomized algorithm for the closest-pair problem

Journal of Algorithms
The space complexity of approximating the frequency moments

Journal of Computer and System Sciences
Even strongly universal hashing is pretty fast

SODA '00 Proceedings of the eleventh annual ACM-SIAM symposium on Discrete algorithms
Models and issues in data stream systems

Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Data streams: algorithms and applications

SODA '03 Proceedings of the fourteenth annual ACM-SIAM symposium on Discrete algorithms
Finding Frequent Items in Data Streams

ICALP '02 Proceedings of the 29th International Colloquium on Automata, Languages and Programming
Universal Hashing and k-Wise Independent Random Variables via Integer Arithmetic without Primes

STACS '96 Proceedings of the 13th Annual Symposium on Theoretical Aspects of Computer Science
Polynomial Hash Functions Are Reliable (Extended Abstract)

ICALP '92 Proceedings of the 19th International Colloquium on Automata, Languages and Programming
Almost random graphs with simple hash functions

Proceedings of the thirty-fifth annual ACM symposium on Theory of computing
Sketch-based change detection: methods, evaluation, and applications

Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
Combinatorial power in multimedia processors

ACM SIGARCH Computer Architecture News
Tabulation based 4-universal hashing with applications to second moment estimation

SODA '04 Proceedings of the fifteenth annual ACM-SIAM symposium on Discrete algorithms

Sketch-based change detection: methods, evaluation, and applications

Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement
Tabulation based 4-universal hashing with applications to second moment estimation

SODA '04 Proceedings of the fifteenth annual ACM-SIAM symposium on Discrete algorithms
Optimal approximations of the frequency moments of data streams

Proceedings of the thirty-seventh annual ACM symposium on Theory of computing
Asymmetric balanced allocation with simple hash functions

SODA '06 Proceedings of the seventeenth annual ACM-SIAM symposium on Discrete algorithm
Detection and identification of network anomalies using sketch subspaces

Proceedings of the 6th ACM SIGCOMM conference on Internet measurement
High-throughput sketch update on a low-power stream processor

Proceedings of the 2006 ACM/IEEE symposium on Architecture for networking and communications systems
Pseudo-random number generation for sketch-based estimations

ACM Transactions on Database Systems (TODS)
Statistical analysis of sketch estimators

Proceedings of the 2007 ACM SIGMOD international conference on Management of data
Linear probing with constant independence

Proceedings of the thirty-ninth annual ACM symposium on Theory of computing
Why simple hash functions work: exploiting the entropy in a data stream

Proceedings of the nineteenth annual ACM-SIAM symposium on Discrete algorithms
Extracting hidden anomalies using sketch and non Gaussian multiresolution statistical detection procedures

Proceedings of the 2007 workshop on Large scale attack defense
Path-quality monitoring in the presence of adversaries

SIGMETRICS '08 Proceedings of the 2008 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Sketches for size of join estimation

ACM Transactions on Database Systems (TODS)
Finding Frequent Items over General Update Streams

SSDBM '08 Proceedings of the 20th international conference on Scientific and Statistical Database Management
Finding frequent items in data streams

Proceedings of the VLDB Endowment
On Estimating Frequency Moments of Data Streams

APPROX '07/RANDOM '07 Proceedings of the 10th International Workshop on Approximation and the 11th International Workshop on Randomization, and Combinatorial Optimization. Algorithms and Techniques
String hashing for linear probing

SODA '09 Proceedings of the twentieth Annual ACM-SIAM Symposium on Discrete Algorithms
Finding the frequent items in streams of data

Communications of the ACM - A View of Parallel Computing
Annotations in Data Streams

ICALP '09 Proceedings of the 36th International Colloquium on Automata, Languages and Programming: Part I
Flooding attacks detection and victim identification over high speed networks

GIIS'09 Proceedings of the Second international conference on Global Information Infrastructure Symposium
b-Bit minwise hashing

Proceedings of the 19th international conference on World wide web
HiFIND: A high-speed flow-level intrusion detection approach with DoS resiliency

Computer Networks: The International Journal of Computer and Telecommunications Networking
A sparse Johnson: Lindenstrauss transform

Proceedings of the forty-second ACM symposium on Theory of computing
Fast Manhattan sketches in data streams

Proceedings of the twenty-ninth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Sketch-based SIP flooding detection using Hellinger distance

GLOBECOM'09 Proceedings of the 28th IEEE conference on Global telecommunications
On the use of sketches and wavelet analysis for network anomaly detection

Proceedings of the 6th International Wireless Communications and Mobile Computing Conference
A scalable, efficient and informative approach for anomaly-based intrusion detection systems: theory and practice

International Journal of Network Management
On the k-independence required by linear probing and minwise independence

ICALP'10 Proceedings of the 37th international colloquium conference on Automata, languages and programming
Note: On perfect hashing of numbers with sparse digit representation via multiplication by a constant

Discrete Applied Mathematics
The power of simple tabulation hashing

Proceedings of the forty-third annual ACM symposium on Theory of computing
Near-optimal private approximation protocols via a black box transformation

Proceedings of the forty-third annual ACM symposium on Theory of computing
Fast moment estimation in data streams in optimal space

Proceedings of the forty-third annual ACM symposium on Theory of computing
Subspace embeddings for the L1-norm with applications

Proceedings of the forty-third annual ACM symposium on Theory of computing
Linear Probing with 5-wise Independence

SIAM Review
Combining wavelet analysis and information theory for network anomaly detection

Proceedings of the 4th International Symposium on Applied Sciences in Biomedical and Communication Technologies
Sparser Johnson-Lindenstrauss transforms

Proceedings of the twenty-third annual ACM-SIAM symposium on Discrete Algorithms
The universality of iterated hashing over variable-length strings

Discrete Applied Mathematics
Optimal bounds for Johnson-Lindenstrauss transforms and streaming problems with sub-constant error

Proceedings of the twenty-second annual ACM-SIAM symposium on Discrete Algorithms
Practical algorithms for tracking database join sizes

FSTTCS '05 Proceedings of the 25th international conference on Foundations of Software Technology and Theoretical Computer Science
Deterministic parallel random-number generation for dynamic-multithreading platforms

Proceedings of the 17th ACM SIGPLAN symposium on Principles and Practice of Parallel Programming
Detecting anomalies in backbone network traffic: a performance comparison among several change detection methods

International Journal of Sensor Networks
Independence of tabulation-based hash classes

LATIN'12 Proceedings of the 10th Latin American international conference on Theoretical Informatics
Tabulation-Based 5-Independent Hashing with Applications to Linear Probing and Second Moment Estimation

SIAM Journal on Computing
Synopses for Massive Data: Samples, Histograms, Wavelets, Sketches

Foundations and Trends in Databases
Explicit and efficient hash families suffice for cuckoo hashing with a stash

ESA'12 Proceedings of the 20th Annual European conference on Algorithms
Optimal Bounds for Johnson-Lindenstrauss Transforms and Streaming Problems with Subconstant Error

ACM Transactions on Algorithms (TALG) - Special Issue on SODA'11
Low rank approximation and regression in input sparsity time

Proceedings of the forty-fifth annual ACM symposium on Theory of computing
A methodological overview on anomaly detection

DataTraffic Monitoring and Analysis

Quantified Score

Hi-index	0.01

Visualization

Abstract

We show that 4-universal hashing can be implemented efficiently using tabulated 4-universal hashing for characters, gaining a factor of 5 in speed over the fastest existing methods. We also consider generalization to k-universal hashing, and as a prime application, we consider the approximation of the second moment of a data stream.