Predicting intermediate storage performance for workflow applications

Authors:
Lauro Beltrao Costa;Samer Al-Kiswany;Abmar Barros;Hao Yang;Matei Ripeanu
Affiliations:
University of British Columbia;University of British Columbia;Universidade Federal de Campina Grande;University of British Columbia;University of British Columbia
Venue:
PDSW '13 Proceedings of the 8th Parallel Data Storage Workshop
Year:
2013

Citing 13
Cited 0

PVFS: A Parallel Virtual File System for Linux Clusters

Linux Journal
Using moldability to improve the performance of supercomputer jobs

Journal of Parallel and Distributed Computing
Hippodrome: Running Circles Around Storage Administration

FAST '02 Proceedings of the Conference on File and Storage Technologies
Quickly finding near-optimal storage designs

ACM Transactions on Computer Systems (TOCS)
Ursa minor: versatile cluster-based storage

FAST'05 Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies - Volume 4
Using utility to provision storage systems

FAST'08 Proceedings of the 6th USENIX Conference on File and Storage Technologies
Informed data distribution selection in a self-predicting storage system

ICAC '06 Proceedings of the 2006 IEEE International Conference on Autonomic Computing
Case studies in storage access by loosely coupled petascale applications

Proceedings of the 4th Annual Workshop on Petascale Data Storage
File-access patterns of data-intensive workflow applications and their implications to distributed filesystems

Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
Assessing data deduplication trade-offs from an energy and performance perspective

IGCC '11 Proceedings of the 2011 International Green Computing Conference and Workshops
Swift: A language for distributed parallel scripting

Parallel Computing
A Workflow-Aware Storage System: An Opportunity Study

CCGRID '12 Proceedings of the 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)
Design and analysis of data management in scalable parallel scripting

SC '12 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis

Quantified Score

Hi-index	0.00

Visualization

Abstract

System configuration decisions for I/O-intensive workflow applications can be complex even for expert users. Users face decisions to configure several parameters optimally (e.g., replication level, chunk size, number of storage node) - each having an impact on overall application performance. This paper presents our progress on addressing the problem of supporting storage system configuration decisions for workflow applications. Our approach accelerates the exploration of the configuration space based on a low-cost performance predictor that estimates turn-around time of a workflow application in a given setup. Our evaluation shows that the predictor is effective in identifying the desired system configuration, and it is lightweight using 2000-5000× less resources (machines × time) than running the actual benchmarks.