SIAM Journal on Computing
Journal of Parallel and Distributed Computing - Special issue on parallel evolutionary computing
Dynamic mapping of a class of independent tasks onto heterogeneous computing systems
Journal of Parallel and Distributed Computing - Special issue on software support for distributed computing
Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing
IEEE Transactions on Parallel and Distributed Systems
Continuous queries over data streams
ACM SIGMOD Record
Distributed and Parallel Databases
Action Port Model: A Mixed Paradigm Conceptual Workflow Modeling Language
COOPIS '98 Proceedings of the 3rd IFCIS International Conference on Cooperative Information Systems
TelegraphCQ: continuous dataflow processing
Proceedings of the 2003 ACM SIGMOD international conference on Management of data
Exact algorithms for NP-hard problems: a survey
Combinatorial optimization - Eureka, you shrink!
A Framework and Ontology for Dynamic Web Services Selection
IEEE Internet Computing
QoS Aggregation for Web Service Composition using Workflow Patterns
EDOC '04 Proceedings of the Enterprise Distributed Object Computing Conference, Eighth IEEE International
Dynamic scheduling of scientific workflow applications on the grid: a case study
Proceedings of the 2005 ACM symposium on Applied computing
Specification and validation of process constraints for flexible workflows
Information Systems
A taxonomy of scientific workflow systems for grid computing
ACM SIGMOD Record
Cost-Based Scheduling of Scientific Workflow Application on Utility Grids
E-SCIENCE '05 Proceedings of the First International Conference on e-Science and Grid Computing
Scientific workflow management and the Kepler system: Research Articles
Concurrency and Computation: Practice & Experience - Workflow in Grid Systems
Workflows for e-Science: Scientific Workflows for Grids
Workflows for e-Science: Scientific Workflows for Grids
Efficient algorithms for Web services selection with end-to-end QoS constraints
ACM Transactions on the Web (TWEB)
SPC: a distributed, scalable platform for data mining
Proceedings of the 4th international workshop on Data mining standards, services and platforms
Scientific Programming - Scientific Workflows
The Trident Scientific Workflow Workbench
ESCIENCE '08 Proceedings of the 2008 Fourth IEEE International Conference on eScience
Heuristics for the variable sized bin-packing problem
Computers and Operations Research
Scientific workflow: a survey and research directions
PPAM'07 Proceedings of the 7th international conference on Parallel processing and applied mathematics
IBM infosphere streams for scalable, real-time, intelligent transportation services
Proceedings of the 2010 ACM SIGMOD International Conference on Management of data
Spark: cluster computing with working sets
HotCloud'10 Proceedings of the 2nd USENIX conference on Hot topics in cloud computing
IEEE Transactions on Services Computing
Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud
CLOUDCOM '10 Proceedings of the 2010 IEEE Second International Conference on Cloud Computing Technology and Science
S4: Distributed Stream Computing Platform
ICDMW '10 Proceedings of the 2010 IEEE International Conference on Data Mining Workshops
WS-Aggregation: distributed aggregation of web services data
Proceedings of the 2011 ACM Symposium on Applied Computing
Nova: continuous Pig/Hadoop workflows
Proceedings of the 2011 ACM SIGMOD International Conference on Management of data
Performance Analysis of Cloud Computing Services for Many-Tasks Scientific Computing
IEEE Transactions on Parallel and Distributed Systems
On the Performance Variability of Production Cloud Services
CCGRID '11 Proceedings of the 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing
Esc: Towards an Elastic Stream Computing Platform for the Cloud
CLOUD '11 Proceedings of the 2011 IEEE 4th International Conference on Cloud Computing
Elastic Stream Computing with Clouds
CLOUD '11 Proceedings of the 2011 IEEE 4th International Conference on Cloud Computing
Efficient dynamic operator placement in a locally distributed continuous query system
ODBASE'06/OTM'06 Proceedings of the 2006 Confederated international conference on On the Move to Meaningful Internet Systems: CoopIS, DOA, GADA, and ODBASE - Volume Part I
End-to-End QoS on Shared Clouds for Highly Dynamic, Large-Scale Sensing Data Streams
CCGRID '12 Proceedings of the 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)
Discretized streams: an efficient and fault-tolerant model for stream processing on large clusters
HotCloud'12 Proceedings of the 4th USENIX conference on Hot Topics in Cloud Ccomputing
Hi-index | 0.00 |
Contemporary continuous dataflow systems use elastic scaling on distributed cloud resources to handle variable data rates and to meet applications' needs while attempting to maximize resource utilization. However, virtualized clouds present an added challenge due to the variability in resource performance -- over time and space -- thereby impacting the application's QoS. Elastic use of cloud resources and their allocation to continuous dataflow tasks need to adapt to such infrastructure dynamism. In this paper, we develop the concept of "dynamic dataflows" as an extension to continuous dataflows that utilizes alternate tasks and allows additional control over the dataflow's cost and QoS. We formalize an optimization problem to perform both deployment and runtime cloud resource management for such dataflows, and define an objective function that allows trade-off between the application's value against resource cost. We present two novel heuristics, local and global, based on the variable sized bin packing heuristics to solve this NP-hard problem. We evaluate the heuristics against a static allocation policy for a dataflow with different data rate profiles that is simulated using VM performance traces from a private cloud data center. The results show that the heuristics are effective in intelligently utilizing cloud elasticity to mitigate the effect of both input data rate and cloud resource performance variabilities on QoS.