A Framework for Building Distributed Data Flow Chains in Clusters

Authors:
Timm M. Steinbeck;Volker Lindenstruth;Dieter Röhrich;Anders Strand Vestbø;Arne Wiebalck
Affiliations:
-;-;-;-;-
Venue:
PARA '02 Proceedings of the 6th International Conference on Applied Parallel Computing Advanced Scientific Computing
Year:
2002

Citing 1
Cited 0

Cluster SMP Nodes with the ATOLL Network: A Look into the Future of System Area Networks

HPCN Europe 2000 Proceedings of the 8th International Conference on High-Performance Computing and Networking

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we present a software framework designed to set up data flow chains in clusters of PCs or workstations. Communication between the components of such a chain is based on the publisher-subscriber-paradigm including bridge components being used to transparently connect components on different nodes. In addition to these bridge components, the framework provides a number of further building blocks to shape the data flow, as well as templates for data sources, sinks, and analysis components. In the bridge components, a class library is used that abstracts from the actual network technology and protocol used and it already exists in versions for TCP/IP and SCI. We will give an overview of the frameworks architecture and present some results obtained in tests of the system.