Distributed stream processing with DUP

Authors:
Kai Christian Bader;Tilo Eißler;Nathan Evans;Chris GauthierDickey;Christian Grothoff;Krista Grothoff;Jeff Keene;Harald Meier;Craig Ritzdorf;Matthew J. Rutherford
Affiliations:
Faculty of Informatics, Technische Universität München;Faculty of Informatics, Technische Universität München;Faculty of Informatics, Technische Universität München;Department of Computer Science, University of Denver;Faculty of Informatics, Technische Universität München;Faculty of Informatics, Technische Universität München;Department of Computer Science, University of Denver;Faculty of Informatics, Technische Universität München;Department of Computer Science, University of Denver;Department of Computer Science, University of Denver
Venue:
NPC'10 Proceedings of the 2010 IFIP international conference on Network and parallel computing
Year:
2010

Citing 14
Cited 1

Linda in context

Communications of the ACM
Coordination languages and their significance

Communications of the ACM
Bounded scheduling of process networks

Bounded scheduling of process networks
Using grep: Moving from DOS? Discover the power of this Linux utility

Linux Journal
Sed and AWK

Sed and AWK
StreamIt: A Language for Streaming Applications

CC '02 Proceedings of the 11th International Conference on Compiler Construction
YODA: selecting signature oligonucleotides

Bioinformatics
Dryad: distributed data-parallel programs from sequential building blocks

Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007
SPC: a distributed, scalable platform for data mining

Proceedings of the 4th international workshop on Data mining standards, services and platforms
Streamflex: high-throughput stream programming in java

Proceedings of the 22nd annual ACM SIGPLAN conference on Object-oriented programming systems and applications
FastForward for efficient pipeline parallelism: a cache-optimized concurrent lock-free queue

Proceedings of the 13th ACM SIGPLAN Symposium on Principles and practice of parallel programming
Optimization principles and application performance evaluation of a multithreaded GPU using CUDA

Proceedings of the 13th ACM SIGPLAN Symposium on Principles and practice of parallel programming
NVIDIA Tesla: A Unified Graphics and Computing Architecture

IEEE Micro
Orchestrating the execution of stream programs on multicore platforms

Proceedings of the 2008 ACM SIGPLAN conference on Programming language design and implementation

Efficient relaxed search in hierarchically clustered sequence datasets

Journal of Experimental Algorithmics (JEA)

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper introduces the DUP System, a simple framework for parallel stream processing. The DUP System enables developers to compose applications from stages written in almost any programming language and to run distributed streaming applications across all POSIXcompatible platforms. Parallel applications written with the DUP System do not suffer from many of the problems that exist in traditional parallel languages. The DUP System includes a range of simple stages that serve as general-purpose building blocks for larger applications. This work describes the DUP Assembly language, the DUP architecture and some of the stages included in the DUP run-time library. We then present our experiences with parallelizing and distributing the ARB project, a package of tools for RNA/DNA sequence database handling and analysis.