Semantic based data collection for large scale cloud systems

  • Authors:
  • Jonathan Stuart Ward;Adam Barker

  • Affiliations:
  • University of St Andrews, St Andrews, Fife, Scotland Uk;University of St Andrews, St Andrews, Fife, Scotland Uk

  • Venue:
  • Proceedings of the fifth international workshop on Data-Intensive Distributed Computing Date
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

Current tools for monitoring cloud systems are designed for physical servers and are not intended to handle rapid elasticity or dynamic behaviour while operating at scale. Though current monitoring tools can be applied to small cloud systems, the volume of data and computational overhead associated with their operation render them unsuitable for large scale cloud deployments. The metrics obtained by current solutions also lack a machine readable structure, limiting the ability of both software and humans to interpret the data. As cloud adoption continues, the scale and complexity of cloud systems will present significant challenges to current tools. This paper proposes a scalable distributed data collection system which forms the basis of a cloud monitoring system. Utilising technologies from the semantic web, our architecture generates a machine readable overview of a cloud system without the need for an additional dedicated monitoring system. We present an exemplar implementation of our architecture written using the Python programming language and perform an evaluation demonstrating its ability to provide scalable data collection services fit for cloud computing