Efficient mid-query re-optimization of sub-optimal query execution plans
SIGMOD '98 Proceedings of the 1998 ACM SIGMOD international conference on Management of data
Compiler optimization of dynamic data distributions for distributed-memory multicomputers
Compiler optimizations for scalable parallel systems
Modern Compiler Implementation in Java
Modern Compiler Implementation in Java
Cross-Platform Performance Prediction of Parallel Applications Using Partial Execution
SC '05 Proceedings of the 2005 ACM/IEEE conference on Supercomputing
The CQL continuous query language: semantic foundations and query execution
The VLDB Journal — The International Journal on Very Large Data Bases
Cost-based query transformation in Oracle
VLDB '06 Proceedings of the 32nd international conference on Very large data bases
Performance prediction with skeletons
Cluster Computing
SPADE: the system s declarative stream processing engine
Proceedings of the 2008 ACM SIGMOD international conference on Management of data
A novel graph based approach for automatic composition of high quality grid workflows
Proceedings of the 18th ACM international symposium on High performance distributed computing
Machine Learning: An Algorithmic Perspective
Machine Learning: An Algorithmic Perspective
A code generation approach to optimizing high-performance distributed data stream processing
Proceedings of the 18th ACM conference on Information and knowledge management
Enhanced subquery optimizations in Oracle
Proceedings of the VLDB Endowment
Towards automatic optimization of MapReduce programs
Proceedings of the 1st ACM symposium on Cloud computing
COLA: optimizing stream processing applications via graph partitioning
Middleware'09 Proceedings of the ACM/IFIP/USENIX 10th international conference on Middleware
Scalable performance of system S for extract-transform-load processing
Proceedings of the 3rd Annual Haifa Experimental Systems Conference
Workload characterization for operator-based distributed stream processing applications
Proceedings of the Fourth ACM International Conference on Distributed Event-Based Systems
Towards optimising distributed data streaming graphs using parallel streams
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
Partitioning streaming parallelism for multi-cores: a machine learning based approach
Proceedings of the 19th international conference on Parallel architectures and compilation techniques
OSDI'08 Proceedings of the 8th USENIX conference on Operating systems design and implementation
Federated enactment of workflow patterns
EuroPar'10 Proceedings of the 16th international Euro-Par conference on Parallel processing: Part I
S4: Distributed Stream Computing Platform
ICDMW '10 Proceedings of the 2010 IEEE International Conference on Data Mining Workshops
Query optimization techniques for partitioned tables
Proceedings of the 2011 ACM SIGMOD International Conference on Management of data
Loop transformation recipes for code generation and auto-tuning
LCPC'09 Proceedings of the 22nd international conference on Languages and Compilers for Parallel Computing
A performance study on operator-based stream processing systems
IISWC '11 Proceedings of the 2011 IEEE International Symposium on Workload Characterization
Automatic optimization of stream programs via source program operator graph transformations
Distributed and Parallel Databases
| Hi-index | 0.00 |
Stream programs have to be crafted carefully to maximize the performance gain that can be obtained from stream processing environments. Manual fine tuning of a stream program is a very difficult process which requires considerable amount of programmer time and expertise. In this paper we present Hirundo, which is a mechanism for automatically generating optimized stream programs that are tailored for the environment they run. Hirundo analyzes, identifies the structure of a stream program, and transforms it to many different sample programs with same semantics using the notions of Tri-Operator Transformation, Transformer Blocks, and Operator Blocks Fusion. Then it uses empirical optimization information to identify a small subset of generated sample programs that could deliver high performance. It runs the selected sample programs in the run-time environment for a short period of time to obtain their performance information. Hirundo utilizes these information to output a ranked list of optimized stream programs that are tailored for a particular run-time environment. Hirundo has been developed using Python as a prototype application for optimizing SPADE programs, which run on System S stream processing run-time. Using three example real world stream processing applications we demonstrate effectiveness of our approach, and discuss how well it generalizes for automatic stream program performance optimization.