Randomized algorithms
Maintaining Stream Statistics over Sliding Windows
SIAM Journal on Computing
Continuously Maintaining Quantile Summaries of the Most Recent N Elements over a Data Stream
ICDE '04 Proceedings of the 20th International Conference on Data Engineering
Effective Computation of Biased Quantiles over Data Streams
ICDE '05 Proceedings of the 21st International Conference on Data Engineering
An improved data stream summary: the count-min sketch and its applications
Journal of Algorithms
Supporting sliding window queries for continuous data streams
SSDBM '03 Proceedings of the 15th International Conference on Scientific and Statistical Database Management
StatStream: statistical monitoring of thousands of data streams in real time
VLDB '02 Proceedings of the 28th international conference on Very Large Data Bases
Hi-index | 0.00 |
Given a data stream of numerical data elements generated from multiple sources, we consider the problem of maintaining the sum of the elements for each data source over a sliding window of the data stream. The difficulties of the problem come from two parts. One is the number of data sources and the other is the number of elements in the sliding window. For massive data sources, we need a significant number of counters to maintain the sum for each data source, while for a large number of data elements in the sliding window, we need a huge space to keep all of them. We propose two methods, which shares the counters efficiently and merge the data elements systematically so that we are able to estimate the sums using a concise data structure. Two parameters, ε and δ, are needed to construct the data structure. ε controls the bounds of the estimate and δ represents the confidence level that the estimate is within the bounds. The estimates of both methods are proven to be bounded within a factor of ε at 1-δ probability.