Executing multiple pipelined data analysis operations in the grid

Authors:
Matthew Spencer;Renato Ferreira;Michael Beynon;Tahsin Kurc;Umit Catalyurek;Alan Sussman;Joel Saltz
Affiliations:
The Ohio State University, Columbus, OH;The Ohio State University, Columbus, OH;University of Maryland, College Park, MD;The Ohio State University, Columbus, OH;The Ohio State University, Columbus, OH;University of Maryland, College Park, MD;The Ohio State University, Columbus, OH
Venue:
Proceedings of the 2002 ACM/IEEE conference on Supercomputing
Year:
2002

Citing 30
Cited 13

Marching cubes: A high resolution 3D surface construction algorithm

SIGGRAPH '87 Proceedings of the 14th annual conference on Computer graphics and interactive techniques
Determining average program execution times and their variance

PLDI '89 Proceedings of the ACM SIGPLAN 1989 Conference on Programming language design and implementation
Supporting dynamic data structures on distributed-memory machines

ACM Transactions on Programming Languages and Systems (TOPLAS)
Monte Carlo simulation of neuro-transmitter release using MCell, a general simulator of cellular physiological processes

CNS '97 Proceedings of the sixth annual conference on Computational neuroscience : trends in research, 1998: trends in research, 1998
The network weather service: a distributed resource performance forecasting service for metacomputing

Future Generation Computer Systems - Special issue on metacomputing
Static scheduling algorithms for allocating directed task graphs to multiprocessors

ACM Computing Surveys (CSUR)
Expressing and enforcing distributed resource sharing agreements

Proceedings of the 2000 ACM/IEEE conference on Supercomputing
A comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems

Journal of Parallel and Distributed Computing
Distributed processing of very large datasets with DataCutter

Parallel Computing - Clusters and computational grids for scientific computing
Processing large-scale multi-dimensional data in parallel and distributed environments

Parallel Computing - Parallel data-intensive algorithms and applications
Efficient Manipulation of Large Datasets on Heterogeneous Storage Systems

IPDPS '02 Proceedings of the 16th International Parallel and Distributed Processing Symposium
Modelling Photochemical Pollution using Parallel and Distributed Computing Platforms

PARLE '94 Proceedings of the 6th International PARLE Conference on Parallel Architectures and Languages Europe
Replica Selection in the Globus Data Grid

CCGRID '01 Proceedings of the 1st International Symposium on Cluster Computing and the Grid
Armada: A Parallel File System for Computational Grids

CCGRID '01 Proceedings of the 1st International Symposium on Cluster Computing and the Grid
Mapping heterogeneous task graphs onto heterogeneous system graphs

HCW '97 Proceedings of the 6th Heterogeneous Computing Workshop (HCW '97)
A Dynamic Matching and Scheduling Algorithm for Heterogeneous Computing Systems

HCW '98 Proceedings of the Seventh Heterogeneous Computing Workshop
Dynamic, Competitive Scheduling of Multiple DAGs in a Distributed Heterogeneous Environment

HCW '98 Proceedings of the Seventh Heterogeneous Computing Workshop
Task Scheduling Algorithms for Heterogeneous Processors

HCW '99 Proceedings of the Eighth Heterogeneous Computing Workshop
Design of a Framework for Data-Intensive Wide-Area Applications

HCW '00 Proceedings of the 9th Heterogeneous Computing Workshop
Run-time Support for Scheduling Parallel Applications in Heterogeneous NOWs

HPDC '97 Proceedings of the 6th IEEE International Symposium on High Performance Distributed Computing
Overview of a Performance Evaluation System for Global Computing Scheduling Algorithms

HPDC '99 Proceedings of the 8th IEEE International Symposium on High Performance Distributed Computing
dQUOB: Managing Large Data Flows Using Dynamic Embedded Queries

HPDC '00 Proceedings of the 9th IEEE International Symposium on High Performance Distributed Computing
Secure, Efficient Data Transport and Replica Management for High-Performance Data-Intensive Computing

MSS '01 Proceedings of the Eighteenth IEEE Symposium on Mass Storage Systems and Technologies
ACDS: Adapting Computational Data Streams for High Performance

IPDPS '00 Proceedings of the 14th International Symposium on Parallel and Distributed Processing
Optimizing Retrieval and Processing of Multi-Dimensional Scientific Datasets

IPDPS '00 Proceedings of the 14th International Symposium on Parallel and Distributed Processing
Parallel Performance Study of Monte Carlo Photon Transport Code on Shared-, Distributed-, and Distributed-Shared-Memory Architectures

IPDPS '00 Proceedings of the 14th International Symposium on Parallel and Distributed Processing
Grid Information Services for Distributed Resource Sharing

HPDC '01 Proceedings of the 10th IEEE International Symposium on High Performance Distributed Computing
A Study of Deadline Scheduling for Client-Server Systems on the Computational Grid

HPDC '01 Proceedings of the 10th IEEE International Symposium on High Performance Distributed Computing
Supporting data intensive applications in a heterogeneous environment

Supporting data intensive applications in a heterogeneous environment
Supporting data intensive applications in a heterogeneous environment

Supporting data intensive applications in a heterogeneous environment

Use of PVFS for Efficient Execution of Jobs with Pipeline-Shared I/O

GRID '04 Proceedings of the 5th IEEE/ACM International Workshop on Grid Computing
A Realistic Network/Application Model for Scheduling Divisible Loads on Large-Scale Platforms

IPDPS '05 Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Papers - Volume 01
A Services Oriented Framework for Next Generation Data Analysis Centers

IPDPS '05 Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Workshop 10 - Volume 11
Improving Data Access for Computational Grid Applications

Cluster Computing
Mapping pipeline skeletons onto heterogeneous platforms

Journal of Parallel and Distributed Computing
Multi-Criteria Scheduling of Pipeline Workflows (and Application To the JPEG Encoder)

International Journal of High Performance Computing Applications
Placing pipeline stages on a Grid: Single path and multipath pipeline execution

Future Generation Computer Systems
Computing the throughput of probabilistic and replicated streaming applications

Proceedings of the twenty-second annual ACM symposium on Parallelism in algorithms and architectures
Optimizing latency and throughput of application workflows on clusters

Parallel Computing
Toward optimizing latency under throughput constraints for application workflows on clusters

Euro-Par'07 Proceedings of the 13th international Euro-Par conference on Parallel Processing
A Novel Energy Saving Approach through Mobile Collaborative Computing Systems

International Journal of Handheld Computing Research
A survey of pipelined workflow scheduling: Models and algorithms

ACM Computing Surveys (CSUR)
Multi-objective exploitation of pipeline parallelism using clustering, replication and duplication in embedded multi-core systems

Journal of Systems Architecture: the EUROMICRO Journal

Quantified Score

Hi-index	0.00

Visualization

Abstract

Processing of data in many data analysis applications can be represented as an acyclic, coarse grain data flow, from data sources to the client. This paper is concerned with scheduling of multiple data analysis operations, each of which is represented as a pipelined chain of processing on data. We define the scheduling problem for effectively placing components onto Grid resources, and propose two scheduling algorithms. Experimental results are presented using a visualization application.