A general approach to data-intensive computing using the Meandre component-based framework

Authors:
Bernie Ács;Xavier Llorà;Loretta Auvil;Boris Capitanu;David Tcheng;Mike Haberman;Limin Dong;Tim Wentling;Michael Welge
Affiliations:
University of Illinois at Urbana-Champaign, Urbana, IL;University of Illinois at Urbana-Champaign, Urbana, IL;University of Illinois at Urbana-Champaign, Urbana, IL;University of Illinois at Urbana-Champaign, Urbana, IL;University of Illinois at Urbana-Champaign, Urbana, IL;University of Illinois at Urbana-Champaign, Urbana, IL;University of Illinois at Urbana-Champaign, Urbana, IL;University of Illinois at Urbana-Champaign, Urbana, IL;University of Illinois at Urbana-Champaign, Urbana, IL
Venue:
Proceedings of the 1st International Workshop on Workflow Approaches to New Data-centric Science
Year:
2010

Citing 4
Cited 2

Design of a Framework for Data-Intensive Wide-Area Applications

HCW '00 Proceedings of the 9th Heterogeneous Computing Workshop
The virtual data grid: a new model and architecture for data-intensive collaboration

SSDBM '03 Proceedings of the 15th International Conference on Scientific and Statistical Database Management
MapReduce: simplified data processing on large clusters

OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Meandre: Semantic-Driven Data-Intensive Flows in the Clouds

ESCIENCE '08 Proceedings of the 2008 Fourth IEEE International Conference on eScience

Meandre data-intensive application infrastructure: extreme scalability for cloud and/or grid computing

JSAI-isAI'10 Proceedings of the 2010 international conference on New Frontiers in Artificial Intelligence
The demand for consistent web-based workflow editors

WORKS '13 Proceedings of the 8th Workshop on Workflows in Support of Large-Scale Science

Quantified Score

Hi-index	0.00

Visualization

Abstract

Data deluge is the norm for research as the volumes of data grow every day. Another complication is the growing number of places and ways that the data needs to be accessed. This leads to a common requirement to integrate a variety from data source and providers to work with the informational resources. This paper describes a general approach to data-intensive computing using a component-based framework. Using the Meandre component-based framework to provided the basis for testing the hypothesis; that a general approach to data-intensive computing integration could be accomplished by creating a family of specialized data-flow components that interface with a given API specification and/or protocols. This inherent functional ability allows the dataflow developers to logically create applications, web services, and/or to directly integrate services provided by other applications or servers. The Fedora Commons Repository Experiment provides two cases that demonstrate how the component-based framework for data-intensive computing can map functional API specifications to discrete functional components that may be used in any number of data flows. This paper presents an overview of the Meandre's anatomy and some functional features it provides for components, it describes the experiment and examines the results that affirm that the approach is viable with the potential to be broadly applied to other API specifications that can then be pooled and combined with other component collections by flow developers to create integrated, interactive, and interoperable application and/or services.