Traversal recursion: a practical approach to supporting recursive applications
SIGMOD '86 Proceedings of the 1986 ACM SIGMOD international conference on Management of data
Transition network grammars for natural language analysis
Communications of the ACM
Aggregation and Relevance in Deductive Databases
VLDB '91 Proceedings of the 17th International Conference on Very Large Data Bases
Exploiting Punctuation Semantics in Continuous Data Streams
IEEE Transactions on Knowledge and Data Engineering
Nile: A Query Processing Engine for Data Streams
ICDE '04 Proceedings of the 20th International Conference on Data Engineering
Flexible time management in data stream systems
PODS '04 Proceedings of the twenty-third ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Semantics and evaluation techniques for window aggregates in data streams
Proceedings of the 2005 ACM SIGMOD international conference on Management of data
Declarative routing: extensible routing with declarative queries
Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communications
A heartbeat mechanism and its application in gigascope
VLDB '05 Proceedings of the 31st international conference on Very large data bases
Implementing declarative overlays
Proceedings of the twentieth ACM symposium on Operating systems principles
Declarative networking: language, execution and optimization
Proceedings of the 2006 ACM SIGMOD international conference on Management of data
High-performance complex event processing over streams
Proceedings of the 2006 ACM SIGMOD international conference on Management of data
Using queries for distributed monitoring and forensics
Proceedings of the 1st ACM SIGOPS/EuroSys European Conference on Computer Systems 2006
VLDB '07 Proceedings of the 33rd international conference on Very large data bases
Efficient pattern matching over event streams
Proceedings of the 2008 ACM SIGMOD international conference on Management of data
Out-of-order processing: a new architecture for high-performance stream systems
Proceedings of the VLDB Endowment
Evita raced: metacompilation for declarative networks
Proceedings of the VLDB Endowment
On Supporting Kleene Closure over Event Streams
ICDE '08 Proceedings of the 2008 IEEE 24th International Conference on Data Engineering
Recursive Computation of Regions and Connectivity in Networks
ICDE '09 Proceedings of the 2009 IEEE International Conference on Data Engineering
Microsoft CEP server and online behavioral targeting
Proceedings of the VLDB Endowment
Semantics of data streams and operators
ICDT'05 Proceedings of the 10th international conference on Database Theory
An extensible test framework for the Microsoft StreamInsight query processor
Proceedings of the Third International Workshop on Testing Database Systems
Real-time spatio-temporal analytics using Microsoft StreamInsight
Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems
High-performance dynamic pattern matching over disordered streams
Proceedings of the VLDB Endowment
Geospatial stream query processing using Microsoft SQL Server StreamInsight
Proceedings of the VLDB Endowment
Active complex event processing over event streams
Proceedings of the VLDB Endowment
An extensibility approach for spatio-temporal stream processing using microsoft streaminsight
SSTD'11 Proceedings of the 12th international conference on Advances in spatial and temporal databases
Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles
ACM SIGOPS 24th Symposium on Operating Systems Principles
Naiad: a timely dataflow system
Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles
Hi-index | 0.01 |
Multiple researchers have proposed cyclic query plans for evaluating iterative queries over streams or rapidly changing input. The Declarative Networking community uses cyclic plans to evaluate Datalog programs that track reachability and other graph traversals on networks. Cyclic query plans can also evaluate pattern-matching and other queries based on event sequences. An issue with cyclic queries over dynamic inputs is knowing when the query result has progressed to a certain point in the input, since the number of iterations is data dependent. One option is a "strictly staged" computation, where the query plan quiesces between inputs. This option introduces significant latency, and may also "underload" inter-operator buffers. An alternative is to settle for soft guarantees, such as "eventual consistency". Such imprecision can make it difficult, for example, to know when to purge state from stateful operators. We propose a third option in which cyclic queries run continuously, but detect progress "on the fly" by means of a Flying Fixed-Point (FFP) operator. FFP sits on the cyclic loop and circulates speculative predictions on forward progress, which it then validates. FFP is always able to track progress for a class of queries we term strongly convergent. A key advantage of FFP is that it works with existing algebra operators, thereby inheriting their capabilities, such as windowing and dealing with out-of-order input. Also, for stream systems that explicitly model input-event lifetimes, we know exactly which values are in the query result at each point in time. A key implementation decision is the method for speculating. Using the high-water mark of data events minimizes the number of speculative punctuations. Probing operators on the cyclic loop to determine their external progress circulates many more speculative messages, but tracks actual output progress more closely. We show how a hybrid approach limits predictions while coming close the progress-tracking ability of Probing.