Experiences with predicting resource performance on-line in computational grid settings

Authors:
Rich Wolski
Affiliations:
University of California, Santa Barbara, Santa Barbara, California
Venue:
ACM SIGMETRICS Performance Evaluation Review
Year:
2003

Citing 22
Cited 41

On the self-similar nature of Ethernet traffic

SIGCOMM '93 Conference proceedings on Communications architectures, protocols and applications
Wide area traffic: the failure of Poisson modeling

IEEE/ACM Transactions on Networking (TON)
Analyzing stability in wide-area network performance

SIGMETRICS '97 Proceedings of the 1997 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Why we don't know how to simulate the Internet

Proceedings of the 29th conference on Winter simulation
Application level scheduling of gene sequence comparison on metacomputers

ICS '98 Proceedings of the 12th international conference on Supercomputing
The grid: blueprint for a new computing infrastructure

The grid: blueprint for a new computing infrastructure
Using pathchar to estimate Internet link characteristics

SIGMETRICS '99 Proceedings of the 1999 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
The network weather service: a distributed resource performance forecasting service for metacomputing

Future Generation Computer Systems - Special issue on metacomputing
Application-level scheduling on distributed heterogeneous networks

Supercomputing '96 Proceedings of the 1996 ACM/IEEE conference on Supercomputing
The AppLeS parameter sweep template: user-level middleware for the grid

Proceedings of the 2000 ACM/IEEE conference on Supercomputing
Writing Programs that Run EveryWare on the Computational Grid

IEEE Transactions on Parallel and Distributed Systems
On the constancy of internet path properties

IMW '01 Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement
Numerical libraries and the grid: the GrADS experiments with ScaLAPACK

Proceedings of the 2001 ACM/IEEE conference on Supercomputing
High-performance remote access to climate simulation data: a challenge problem for data grid technologies

Proceedings of the 2001 ACM/IEEE conference on Supercomputing
Dynamically forecasting network performance using the Network Weather Service

Cluster Computing
Predicting the Performance of Wide Area Data Transfers

IPDPS '02 Proceedings of the 16th International Parallel and Distributed Processing Symposium
Multivariate resource performance forecasting in the network weather service

Proceedings of the 2002 ACM/IEEE conference on Supercomputing
NwsAlarm: A Tool for Accurately Detecting Resource Performance Degradation

CCGRID '01 Proceedings of the 1st International Symposium on Cluster Computing and the Grid
Experiments of Network Throughput Measurement and Forecasting Using the Network Weather

CCGRID '02 Proceedings of the 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid
Grid Information Services for Distributed Resource Sharing

HPDC '01 Proceedings of the 10th IEEE International Symposium on High Performance Distributed Computing
Online Prediction of the Running Time of Tasks

HPDC '01 Proceedings of the 10th IEEE International Symposium on High Performance Distributed Computing
The GrADS Project: Software Support for High-Level Grid Application Development

International Journal of High Performance Computing Applications

GridSAT: A Chaff-based Distributed SAT Solver for the Grid

Proceedings of the 2003 ACM/IEEE conference on Supercomputing
The Livny and Plank-Beck Problems: Studies in Data Movement on the Computational Grid

Proceedings of the 2003 ACM/IEEE conference on Supercomputing
Models and Modeling Infrastructures for Global Computational Platforms

IPDPS '05 Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Workshop 10 - Volume 11
Using model trees to characterize computer resource usage

WOSS '04 Proceedings of the 1st ACM SIGSOFT workshop on Self-managed systems
Adaptive On-the-Fly Compression

IEEE Transactions on Parallel and Distributed Systems
On the efficacy, efficiency and emergent behavior of task replication in large distributed systems

Parallel Computing
A prediction method for job runtimes on shared processors: Survey, statistical analysis and new avenues

Performance Evaluation
MOB: zero-configuration high-throughput multicasting for grid applications

Proceedings of the 16th international symposium on High performance distributed computing
Classic Paxos vs. fast Paxos: caveat emptor

HotDep'07 Proceedings of the 3rd workshop on on Hot Topics in System Dependability
NWSLite: A general-purpose, nonparametric prediction utility for embedded systems

ACM Transactions on Embedded Computing Systems (TECS)
Allocation strategies for utilization of space-shared resources in Bag of Tasks grids

Future Generation Computer Systems
Error recovery mechanism for grid-based workflow within SLA context

International Journal of High Performance Computing and Networking
A recovery mechanism for errors caused by a late subjob in a system handling SLA-based Grid workflows

International Journal of Web and Grid Services
Mapping Heavy Communication Grid-Based Workflows Onto Grid Resources Within an SLA Context Using Metaheuristics

International Journal of High Performance Computing Applications
Neural Network-Based Load Prediction for Highly Dynamic Distributed Online Games

Euro-Par '08 Proceedings of the 14th international Euro-Par conference on Parallel Processing
Enhancing Prediction on Non-dedicated Clusters

Euro-Par '08 Proceedings of the 14th international Euro-Par conference on Parallel Processing
A service-oriented monitoring system with a forecasting algorithm of a time sequence-based hybrid model

International Journal of Parallel, Emergent and Distributed Systems - Best Papers from the GCC 2006 Conference
On the Efficacy of Computation Offloading Decision-Making Strategies

International Journal of High Performance Computing Applications
Prediction-based real-time resource provisioning for massively multiplayer online games

Future Generation Computer Systems
Adaptive run-time prediction in heterogeneous environments

Proceedings of the 18th ACM international symposium on High performance distributed computing
Trace-based evaluation of job runtime and queue wait time predictions in grids

Proceedings of the 18th ACM international symposium on High performance distributed computing
Multi-scale Real-Time Grid Monitoring with Job Stream Mining

CCGRID '09 Proceedings of the 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid
Dynamic Real-Time Resource Provisioning for Massively Multiplayer Online Games

PaCT '09 Proceedings of the 10th International Conference on Parallel Computing Technologies
Mixture of ANFIS systems for CPU load prediction in metacomputing environment

Future Generation Computer Systems
A network topology mapping tool for the grid

Proceedings of the 7th International Conference on Frontiers of Information Technology
Managing risks in an open computing environment using mean absolute deviation portfolio optimization

Future Generation Computer Systems
Adaps - A three-phase adaptive prediction system for the run-time of jobs based on user behaviour

Journal of Computer and System Sciences
CPU load prediction using neuro-fuzzy and Bayesian inferences

Neurocomputing
On/off-line prediction applied to job scheduling on non-dedicated NOWs

Journal of Computer Science and Technology - Special issue on natural language processing
A multi-strategy collaborative prediction model for the runtime of online tasks in computing cluster/grid

Cluster Computing
Mapping heavy communication workflows onto grid resources within an SLA context

HPCC'06 Proceedings of the Second international conference on High Performance Computing and Communications
An adaptive skeletal task farm for grids

Euro-Par'05 Proceedings of the 11th international Euro-Par conference on Parallel Processing
A dynamic supervising model based on grid environment

KES'05 Proceedings of the 9th international conference on Knowledge-Based Intelligent Information and Engineering Systems - Volume Part II
Bulls-Eye – a resource provisioning service for enterprise distributed real-time and embedded systems

ODBASE'06/OTM'06 Proceedings of the 2006 Confederated international conference on On the Move to Meaningful Internet Systems: CoopIS, DOA, GADA, and ODBASE - Volume Part II
A monitoring architecture for control grids

EGC'05 Proceedings of the 2005 European conference on Advances in Grid Computing
An effective data aggregation based adaptive long term CPU load prediction mechanism on computational grid

Future Generation Computer Systems
An architecture for resource behavior prediction to improve scheduling systems performance on enterprise desktop grids

ADNTIIC'11 Proceedings of the Second international conference on Advances in New Technologies, Interactive Interfaces and Communicability
State-based predictions with self-correction on Enterprise Desktop Grid environments

Journal of Parallel and Distributed Computing
Adaptive monitoring: a framework to adapt passive monitoring using probing

Proceedings of the 8th International Conference on Network and Service Management
Adaptive Online Compression in Clouds--Making Informed Decisions in Virtual Machine Environments

Journal of Grid Computing
Resource utilization prediction: long term network web service traffic

Proceedings of the 2nd annual conference on Research in information technology

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we describe methods for predicting the performance of Computational Grid resources (machines, networks, storage systems, etc.) using computationally inexpensive statistical techniques. The predictions generated in this manner are intended to support adaptive application scheduling in Grid settings, and on-line fault detection. Wedescribe a mixture-of-experts approach to non-parametric, univariate time-series forecasting, and detail the effectiveness of the approach using example data gathered from "production" (i.e. non-experimental) Computational Grid installations.