A relational approach to monitoring complex systems
ACM Transactions on Computer Systems (TOCS)
Continuous queries over append-only databases
SIGMOD '92 Proceedings of the 1992 ACM SIGMOD international conference on Management of data
Eddies: continuously adaptive query processing
SIGMOD '00 Proceedings of the 2000 ACM SIGMOD international conference on Management of data
OceanStore: an architecture for global-scale persistent storage
ASPLOS IX Proceedings of the ninth international conference on Architectural support for programming languages and operating systems
Applying scheduling and tuning to on-line parallel tomography
Proceedings of the 2001 ACM/IEEE conference on Supercomputing
Continual Queries for Internet Scale Event-Driven Information Delivery
IEEE Transactions on Knowledge and Data Engineering
Application-Dependent Dynamic Monitoring of Distributed and Parallel Systems
IEEE Transactions on Parallel and Distributed Systems
Autopilot: Adaptive Control of Distributed Applications
HPDC '98 Proceedings of the 7th IEEE International Symposium on High Performance Distributed Computing
Event Services for High Performance Computing
HPDC '00 Proceedings of the 9th IEEE International Symposium on High Performance Distributed Computing
dQUOB: Managing Large Data Flows Using Dynamic Embedded Queries
HPDC '00 Proceedings of the 9th IEEE International Symposium on High Performance Distributed Computing
SS '95 Proceedings of the 28th Annual Simulation Symposium
Fjording the Stream: An Architecture for Queries Over Streaming Sensor Data
ICDE '02 Proceedings of the 18th International Conference on Data Engineering
Dynamic Querying of Streaming Data with the dQUOB System
IEEE Transactions on Parallel and Distributed Systems
Resource allocation in a middleware for streaming data
MGC '04 Proceedings of the 2nd workshop on Middleware for grid computing
Evaluation of Rate-Based Adaptivity in Asynchronous Data Stream Joins
IPDPS '05 Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Papers - Volume 01
Supporting dynamic migration in tightly coupled grid applications
Proceedings of the 2006 ACM/IEEE conference on Supercomputing
Stream processing in data-driven computational science
GRID '06 Proceedings of the 7th IEEE/ACM International Conference on Grid Computing
Supporting self-adaptation in streaming data mining applications
IPDPS'06 Proceedings of the 20th international conference on Parallel and distributed processing
| Hi-index | 0.00 |
The dQUOB system conceptualization of datastreams as database and its SQL interface to data streams is an intuitive way for users to think about their data needs in a large scale application containing hundreds if not thousands of data streams. Experience with dQUOBhas shown the need for more aggressive memory management to achieve the scalability we desire. This paper addresses the problem with a two-fold solution. The first is replacement of the existing First Come First Served (FCFS) scheduling algorithm with an Earliest Job First(EJF) algorithm which we demonstrate to yield better average service time. The second is an introspection algorithm that sets and adapts the sizes of join windows in response to knowledge acquired at runtime about event rates. In addition to the potential for significant improvements in memory utilization, the algorithm presented here also provides a means by which the user can reason about join window sizes. Wide area measurements demonstrate the adaptive capability required by the introspection technique.