SIGMOD '97 Proceedings of the 1997 ACM SIGMOD international conference on Management of data
Dataflow query execution in a parallel main-memory environment
PDIS '91 Proceedings of the first international conference on Parallel and distributed information systems
Models and issues in data stream systems
Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Approximate join processing over data streams
Proceedings of the 2003 ACM SIGMOD international conference on Management of data
Data Mining Meets Performance Evaluation: Fast Algorithms for Modeling Bursty Traffic
ICDE '02 Proceedings of the 18th International Conference on Data Engineering
Hash-Merge Join: A Non-blocking Join Algorithm for Producing Fast and Early Join Results
ICDE '04 Proceedings of the 20th International Conference on Data Engineering
Caching queues in memory buffers
SODA '04 Proceedings of the fifteenth annual ACM-SIAM symposium on Discrete algorithms
Adaptive Caching for Continuous Queries
ICDE '05 Proceedings of the 21st International Conference on Data Engineering
RPJ: producing fast join results on streams through rate-based optimization
Proceedings of the 2005 ACM SIGMOD international conference on Management of data
Adaptive load shedding for windowed stream joins
Proceedings of the 14th ACM international conference on Information and knowledge management
Run-time operator state spilling for memory intensive long-running queries
Proceedings of the 2006 ACM SIGMOD international conference on Management of data
High-performance complex event processing over streams
Proceedings of the 2006 ACM SIGMOD international conference on Management of data
Window join approximation over data streams with importance semantics
CIKM '06 Proceedings of the 15th ACM international conference on Information and knowledge management
Monitoring streams: a new class of data management applications
VLDB '02 Proceedings of the 28th international conference on Very Large Data Bases
Progressive merge join: a generic and non-blocking sort-based join algorithm
VLDB '02 Proceedings of the 28th international conference on Very Large Data Bases
Maximizing the output rate of multi-way join queries over streaming information sources
VLDB '03 Proceedings of the 29th international conference on Very large data bases - Volume 29
Load shedding in a data stream manager
VLDB '03 Proceedings of the 29th international conference on Very large data bases - Volume 29
Processing sliding window multi-joins in continuous queries over data streams
VLDB '03 Proceedings of the 29th international conference on Very large data bases - Volume 29
Memory-limited execution of windowed stream joins
VLDB '04 Proceedings of the Thirtieth international conference on Very large data bases - Volume 30
Querying sliding windows over online data streams
EDBT'04 Proceedings of the 2004 international conference on Current Trends in Database Technology
| Hi-index | 0.00 |
We consider the problem of processing exact results for sliding window joins over data streams with limited memory. Existing approaches deal with memory limitations by shedding loads, and therefore cannot provide exact or even highly accurate results for sliding window joins over data streams showing time varying rate of data arrivals. We provide an exact window join (EWJ) algorithm incorporating disk storage as an archive. Our algorithm spills window data onto the disk on a periodic basis, refines the output result by properly retrieving the disk resident data, and maximises output rate by employing techniques to manage the memory blocks. The problem of managing the window blocks in memory – similar in nature to the caching issue – captures both the temporal and frequency related properties of the stream arrivals. We provide experimental results demonstrating the performance and effectiveness of the proposed algorithm.