Resource allocation in a middleware for streaming data

Authors:
Liang Chen;Gagan Agrawal
Affiliations:
Ohio State University, Columbus OH;Ohio State University, Columbus OH
Venue:
MGC '04 Proceedings of the 2nd workshop on Middleware for grid computing
Year:
2004

Citing 18
Cited 6

New sampling-based summary statistics for improving approximate query answers

SIGMOD '98 Proceedings of the 1998 ACM SIGMOD international conference on Management of data
Resource management in Legion

Future Generation Computer Systems - Special issue on metacomputing
Rate-based query optimization for streaming information sources

Proceedings of the 2002 ACM SIGMOD international conference on Management of data
Grid Services for Distributed System Integration

Computer
Dynamic Querying of Streaming Data with the dQUOB System

IEEE Transactions on Parallel and Distributed Systems
Issues in data stream management

ACM SIGMOD Record
QoS-Aware Discovery of Wide-Area Distributed Services

CCGRID '01 Proceedings of the 1st International Symposium on Cluster Computing and the Grid
Agent-Based Resource Discovery

HCW '00 Proceedings of the 9th Heterogeneous Computing Workshop
Matchmaking: Distributed Resource Management for High Throughput Computing

HPDC '98 Proceedings of the 7th IEEE International Symposium on High Performance Distributed Computing
Leveraging Run Time Knowledge about Event Rates to Improve Memory Utilization in Wide Area Data Stream Filtering

HPDC '02 Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing
A Peer-to-Peer Approach to Resource Location in Grid Environments

HPDC '02 Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing
Resource Discovery for Dynamic Clusters in Computational Grids

IPDPS '01 Proceedings of the 10th Heterogeneous Computing Workshop â"" HCW 2001 (Workshop 1) - Volume 2
Agents for the Grid: A Comparison with Web Services (Part I: Transport Layer)

CCGRID '02 Proceedings of the 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid
Grid Information Services for Distributed Resource Sharing

HPDC '01 Proceedings of the 10th IEEE International Symposium on High Performance Distributed Computing
GATES: A Grid-Based Middleware for Processing Distributed Data Streams

HPDC '04 Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing
Distributed computing in practice: the Condor experience: Research Articles

Concurrency and Computation: Practice & Experience - Grid Performance
The Anatomy of the Grid: Enabling Scalable Virtual Organizations

International Journal of High Performance Computing Applications
Monitoring streams: a new class of data management applications

VLDB '02 Proceedings of the 28th international conference on Very Large Data Bases

Scaling Properties of Common Statistical Operators for Gridded Datasets

International Journal of High Performance Computing Applications
Integration of sensing and computing in an intelligent decision support system for homeland security defense

Pervasive and Mobile Computing
Server-side parallel data reduction and analysis

GPC'07 Proceedings of the 2nd international conference on Advances in grid and pervasive computing
Optimizing end-to-end performance of data-intensive computing pipelines in heterogeneous network environments

Journal of Parallel and Distributed Computing
Optimizing latency and throughput of application workflows on clusters

Parallel Computing
Latency modeling and minimization for large-scale scientific workflows in distributed network environments

Proceedings of the 44th Annual Simulation Symposium

Quantified Score

Hi-index	0.00

Visualization

Abstract

Increasingly, a number of applications rely on, or can potentially benefit from, analysis and monitoring of data streams. To support processing of streaming data in a grid environment, we have been developing a middleware system called GATES (Grid-based AdapTive Execution on Streams). Our target applications are those involving high volume data streams and requiring distributed processing of data arising from a distributed set of sources. This paper addresses the problem of resource allocation in the GATES system. Though resource discovery and resource allocation have been active topics in grid community, the pipelined processing and real-time constraint required by distributed streaming applications pose new challenges. We present a resource allocation algorithm that is based on minimal spanning trees. We evaluate the algorithm experimentally and demonstrate that it results in configurations that are very close to optimal, and significantly better than most other possible configurations.