A framework to compute statistics of system parameters from very large trace files

Authors:
Naser Ezzati-Jivan;Michel R. Dagenais
Affiliations:
Department of Computer and Software Engineering, Ecole Polytechnique de Montreal, Montreal, Canada;Department of Computer and Software Engineering, Ecole Polytechnique de Montreal, Montreal, Canada
Venue:
ACM SIGOPS Operating Systems Review
Year:
2013

Citing 8
Cited 0

Multidimensional access methods

ACM Computing Surveys (CSUR)
A Tree Based Access Method (TBSAM) for Fast Processing of Aggregate Queries

Proceedings of the Fourth International Conference on Data Engineering
Capturing, indexing, clustering, and retrieving system history

Proceedings of the twentieth ACM symposium on Operating systems principles
Correlating instrumentation data to system states: a building block for automated diagnosis and control

OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Computational Geometry: Algorithms and Applications

Computational Geometry: Algorithms and Applications
Detailed diagnosis in enterprise networks

Proceedings of the ACM SIGCOMM 2009 conference on Data communication
Online System Problem Detection by Mining Patterns of Console Logs

ICDM '09 Proceedings of the 2009 Ninth IEEE International Conference on Data Mining
A stateful approach to generate synthetic events from Kernel traces

Advances in Software Engineering

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we present a framework to compute, store and retrieve statistics of various system metrics from large traces in an efficient way. The proposed framework allows for rapid interactive queries about system metrics values for any given time interval. In the proposed framework, efficient data structures and algorithms are designed to achieve a reasonable query time while utilizing less disk space. A parameter termed granularity degree (GD) is defined to determine the threshold of how often it is required to store the precomputed statistics on disk. The solution supports the hierarchy of system resources and also different granularities of time ranges. We explain the architecture of the framework and show how it can be used to efficiently compute and extract the CPU usage and other system metrics. The importance of the framework and its different applications are shown and evaluated in this paper.