On distributed computation rate optimization for deploying cloud computing programming frameworks

Authors:
Jia Liu;Cathy H. Xia;Ness B. Shroff;Xiaodong Zhang
Affiliations:
The Ohio State University, Columbus, OH, USA;The Ohio State University, Columbus, OH, USA;The Ohio State University, Columbus, OH, USA;The Ohio State University, Columbus, OH, USA
Venue:
ACM SIGMETRICS Performance Evaluation Review
Year:
2013

Citing 12
Cited 0

Dynamic tree embeddings in butterflies and hypercubes

SIAM Journal on Computing
A method for them embedding of arbitrary communication topologies into configurable parallel computers

SAC '98 Proceedings of the 1998 ACM symposium on Applied Computing
Efficient dynamic embeddings of binary trees into hypercubes

Journal of Algorithms
MapReduce: simplified data processing on large clusters

OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Dryad: distributed data-parallel programs from sequential building blocks

Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007
Load shedding and distributed resource control of stream processing networks

Performance Evaluation
Optimistic Synchronization of Parallel Simulations in Cloud Computing Environments

CLOUD '09 Proceedings of the 2009 IEEE International Conference on Cloud Computing
A unified modeling framework for distributed resource allocation of general fork and join processing networks

Proceedings of the ACM SIGMETRICS international conference on Measurement and modeling of computer systems
DOT: a matrix model for analyzing, optimizing and deploying software for big data analytics in distributed systems

Proceedings of the 2nd ACM Symposium on Cloud Computing
Extreme Data-Intensive Scientific Computing

Computing in Science and Engineering
Parallel data processing with MapReduce: a survey

ACM SIGMOD Record
A tutorial on cross-layer optimization in wireless networks

IEEE Journal on Selected Areas in Communications

Quantified Score

Hi-index	0.00

Visualization

Abstract

With the rapidly growing challenges of big data analytics, the need for efficient and distributed algorithms to optimize cloud computing performances is unprecedentedly high. In this paper, we consider how to optimally deploy a cloud computing programming framework (e.g., MapReduce and Dryad) over a given underlying network hardware infrastructure to maximize the end-to-end computation rate and minimize the overall computation and communication costs. The main contributions in this paper are three-fold: i) we develop a new network flow model with a generalized flowconservation law to enable a systematic design of distributed algorithms for computation rate utility maximization problems (CRUM) in cloud computing; ii) based on the network flow model, we reveal key separable properties of the dual functions of Problem CRUM, which further lead to a distributed algorithm design; and iii) we offer important networking insights and meaningful economic interpretations for the proposed algorithm and point out their connections to and distinctions from distributed algorithms design in traditional data communications networks. This paper serves as an important first step towards the development of a theoretical foundation for distributed computation analytics in cloud computing.