Measuring and Understanding User Comfort With Resource Borrowing
HPDC '04 Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing
MOON: MapReduce On Opportunistic eNvironments
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
Nebulas: using distributed voluntary resources to build clouds
HotCloud'09 Proceedings of the 2009 conference on Hot topics in cloud computing
Black-box and gray-box strategies for virtual machine migration
NSDI'07 Proceedings of the 4th USENIX conference on Networked systems design & implementation
No one (cluster) size fits all: automatic cluster sizing for data-intensive analytics
Proceedings of the 2nd ACM Symposium on Cloud Computing
Jockey: guaranteed job latency in data parallel clusters
Proceedings of the 7th ACM european conference on Computer Systems
Resource-aware adaptive scheduling for mapreduce clusters
Middleware'11 Proceedings of the 12th ACM/IFIP/USENIX international conference on Middleware
Orchestrating the deployment of computations in the cloud with conductor
NSDI'12 Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation
| Hi-index | 0.00 |
The popularity of cloud-based interactive computing services (e.g., virtual desktops) brings new management challenges. Each interactive user leaves abundant but fluctuating residual resources while being intolerant to latency, precluding the use of aggressive VM consolidation. In this paper, we present the Resource Harvester for Interactive Clouds (RHIC), an autonomous management framework that harnesses dynamic residual resources aggressively without slowing the harvested interactive services. RHIC builds ad-hoc clusters for running throughput-oriented "background" workloads using a hybrid of residual and dedicated resources. For a given background job, RHIC intelligently discovers/maintains the ideal cluster size and composition, to meet user-specified goals such as cost/energy minimization or deadlines. RHIC employs black-box workload performance modeling, requiring only system-level metrics and incorporating techniques to improve modeling accuracy under bursty and heterogeneous residual resources. Our results show that RHIC finds near-ideal cluster sizes/compositions across a wide range of workload/goal combinations, significantly outperforms alternative approaches, tolerates high instability in the harvested interactive cloud, works with heterogeneous hardware and imposes minimal overhead.