CloudSense: continuous fine-grain cloud monitoring with compressive sensing

Authors:
H. T. Kung;Chit-Kwan Lin;Dario Vlah
Affiliations:
Harvard University, Cambridge, MA;Harvard University, Cambridge, MA;Harvard University, Cambridge, MA
Venue:
HotCloud'11 Proceedings of the 3rd USENIX conference on Hot topics in cloud computing
Year:
2011

Citing 7
Cited 2

Astrolabe: A robust and scalable technology for distributed system monitoring, management, and data mining

ACM Transactions on Computer Systems (TOCS)
Autopilot: automatic data center management

ACM SIGOPS Operating Systems Review - Systems work at Microsoft Research
The cost of a cloud: research problems in data center networks

ACM SIGCOMM Computer Communication Review
Compressive data gathering for large-scale wireless sensor networks

Proceedings of the 15th annual international conference on Mobile computing and networking
SideCar: building programmable datacenter networks without programmable switches

Hotnets-IX Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks
Reining in the outliers in map-reduce clusters using Mantri

OSDI'10 Proceedings of the 9th USENIX conference on Operating systems design and implementation
Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information

IEEE Transactions on Information Theory

QoS monitoring and dynamic trust establishment in the cloud

GPC'12 Proceedings of the 7th international conference on Advances in Grid and Pervasive Computing
Data-Intensive Cloud Computing: Requirements, Expectations, Challenges, and Solutions

Journal of Grid Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Continuous fine-grain status monitoring of a cloud data center enables rapid response to anomalies, but handling the resulting torrent of data poses a significant challenge. As a solution, we propose CloudSense, a new switch design that performs in-network compression of status streams via compressive sensing. Using MapReduce straggler detection as an example of cloud monitoring, we give evidence that CloudSense allows earlier detection of stragglers, since finer-grain status can be reported for a given bandwidth budget. Furthermore, CloudSense showcases the advantage of an intrinsic property of compressive sensing decoding that enables detection of the slowest stragglers first. Finally, CloudSense achieves in-network compression via a low-complexity encoding scheme, which is easy and convenient to implement in a switch. We envision that CloudSense switches could form the foundation of a "compressed status information plane" that is useful for monitoring not only the cloud data center itself, but also the user applications that it hosts.