Parallel database systems: the future of high performance database systems
Communications of the ACM
Eliminating receive livelock in an interrupt-driven kernel
ACM Transactions on Computer Systems (TOCS)
Optimizing Queries with Object Updates
Journal of Intelligent Information Systems - Special issue on functional approach to intelligent information systems
Eddies: continuously adaptive query processing
SIGMOD '00 Proceedings of the 2000 ACM SIGMOD international conference on Management of data
A stream compiler for communication-exposed architectures
Proceedings of the 10th international conference on Architectural support for programming languages and operating systems
StreamIt: A Language for Streaming Applications
CC '02 Proceedings of the 11th International Conference on Compiler Construction
LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation
Proceedings of the international symposium on Code generation and optimization: feedback-directed and runtime optimization
Proceedings of the 13th International Conference on Parallel Architectures and Compilation Techniques
OOPSLA '04 Proceedings of the 19th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications
System support for pervasive applications
ACM Transactions on Computer Systems (TOCS)
Network-Aware Operator Placement for Stream-Processing Systems
ICDE '06 Proceedings of the 22nd International Conference on Data Engineering
Better extensibility through modular syntax
Proceedings of the 2006 ACM SIGPLAN conference on Programming language design and implementation
LINQ: reconciling object, relations and XML in the .NET framework
Proceedings of the 2006 ACM SIGMOD international conference on Management of data
The CQL continuous query language: semantic foundations and query execution
The VLDB Journal — The International Journal on Very Large Data Bases
Exploiting coarse-grained task, data, and pipeline parallelism in stream programs
Proceedings of the 12th international conference on Architectural support for programming languages and operating systems
Interpreting the data: Parallel analysis with Sawzall
Scientific Programming - Dynamic Grids and Worldwide Computing
MapReduce: simplified data processing on large clusters
OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Dryad: distributed data-parallel programs from sequential building blocks
Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007
SPC: a distributed, scalable platform for data mining
Proceedings of the 4th international workshop on Data mining standards, services and platforms
Jeannie: granting java native interface developers their wishes
Proceedings of the 22nd annual ACM SIGPLAN conference on Object-oriented programming systems and applications
XML query optimization in the presence of side effects
Proceedings of the 2008 ACM SIGMOD international conference on Management of data
Pig latin: a not-so-foreign language for data processing
Proceedings of the 2008 ACM SIGMOD international conference on Management of data
SPADE: the system s declarative stream processing engine
Proceedings of the 2008 ACM SIGMOD international conference on Management of data
A lightweight streaming layer for multicore execution
ACM SIGARCH Computer Architecture News
SODA: an optimizing scheduler for large-scale stream-based distributed computer systems
Proceedings of the 9th ACM/IFIP/USENIX International Conference on Middleware
Polyglot: an extensible compiler framework for Java
CC'03 Proceedings of the 12th international conference on Compiler construction
DBPL'07 Proceedings of the 11th international conference on Database programming languages
FlumeJava: easy, efficient data-parallel pipelines
PLDI '10 Proceedings of the 2010 ACM SIGPLAN conference on Programming language design and implementation
COLA: optimizing stream processing applications via graph partitioning
Middleware'09 Proceedings of the ACM/IFIP/USENIX 10th international conference on Middleware
Proceedings of the 19th international conference on Parallel architectures and compilation techniques
OSDI'08 Proceedings of the 8th USENIX conference on Operating systems design and implementation
Lime: a Java-compatible and synthesizable language for heterogeneous architectures
Proceedings of the ACM international conference on Object oriented programming systems languages and applications
CASES '10 Proceedings of the 2010 international conference on Compilers, architectures and synthesis for embedded systems
CIEL: a universal execution engine for distributed data-flow computing
Proceedings of the 8th USENIX conference on Networked systems design and implementation
A universal calculus for stream processing languages
ESOP'10 Proceedings of the 19th European conference on Programming Languages and Systems
Partition and compose: parallel complex event processing
Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems
A catalog of stream processing optimizations
ACM Computing Surveys (CSUR)
| Hi-index | 0.00 |
At one level, this paper is about River, a virtual execution environment for stream processing. Stream processing is a paradigm well-suited for many modern data processing systems that ingest high-volume data streams from the real world, such as audio/video streaming, high-frequency trading, and security monitoring. One attractive property of stream processing is that it lends itself to parallelization on multicores, and even to distribution on clusters when extreme scale is required. Stream processing has been co-evolved by several communities, leading to diverse languages with similar core concepts. Providing a common execution environment reduces language development effort and increases portability. We designed River as a practical realization of Brooklet, a calculus for stream processing. So at another level, this paper is about a journey from theory (the calculus) to practice (the execution environment). The challenge is that, by definition, a calculus abstracts away all but the most central concepts. Hence, there are several research questions in concretizing the missing parts, not to mention a significant engineering effort in implementing them. But the effort is well worth it, because using a calculus as a foundation yields clear semantics and proven correctness results.