Efficiently extracting operational profiles from execution logs using suffix arrays

Authors:
Meiyappan Nagappan;Kesheng Wu;Mladen A. Vouk
Affiliations:
North Carolina State University, Dept. of Computer Science, Raleigh, NC;Lawrence Berkeley National Laboratory, Computational Research Division, Berkeley, CA;North Carolina State University, Dept. of Computer Science, Raleigh, NC
Venue:
ISSRE'09 Proceedings of the 20th IEEE international conference on software reliability engineering
Year:
2009

Citing 15
Cited 1

The operational profile

Handbook of software reliability engineering
Suffix arrays: a new method for on-line string searches

SODA '90 Proceedings of the first annual ACM-SIAM symposium on Discrete algorithms
A methodology for workload characterization of E-commerce sites

Proceedings of the 1st ACM conference on Electronic commerce
Operational Profiles in Software-Reliability Engineering

IEEE Software
Linear-Time Longest-Common-Prefix Computation in Suffix Arrays and Its Applications

CPM '01 Proceedings of the 12th Annual Symposium on Combinatorial Pattern Matching
Automated System Monitoring and Notification With Swatch

LISA '93 Proceedings of the 7th USENIX conference on System administration
A survey of trace exploration tools and techniques

CASCON '04 Proceedings of the 2004 conference of the Centre for Advanced Studies on Collaborative research
Mapping moving landscapes by mining mountains of logs: novel techniques for dependency model generation

VLDB '06 Proceedings of the 32nd international conference on Very large data bases
What Supercomputers Say: A Study of Five System Logs

DSN '07 Proceedings of the 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks
An Industrial Case Study of Customizing Operational Profiles Using Log Compression

Proceedings of the 30th international conference on Software engineering
Fast and Space Efficient Linear Suffix Array Construction

DCC '08 Proceedings of the Data Compression Conference
Abstracting Execution Logs to Execution Events for Enterprise Applications (Short Paper)

QSIC '08 Proceedings of the 2008 The Eighth International Conference on Quality Software
Efficient Operational Profiling of Systems Using Suffix Arrays on Execution Logs

ISSRE '08 Proceedings of the 2008 19th International Symposium on Software Reliability Engineering
A metric for predicting the performance of an application under a growing workload

IBM Systems Journal
An offline approach for whole-program paths analysis using suffix arrays

LCPC'04 Proceedings of the 17th international conference on Languages and Compilers for High Performance Computing

Bridging the divide between software developers and operators using logs

Proceedings of the 34th International Conference on Software Engineering

Quantified Score

Hi-index	0.00

Visualization

Abstract

An important software reliability engineering tool is operational profiles. In this paper we propose a cost effective automated approach for creating second generation operational profiles using execution logs of a software product. Our algorithm parses the execution logs into sequences of events and produces an ordered list of all possible subsequences by constructing a suffix array of the events. The difficulty in using execution logs is that the amount of data that needs to be analyzed is often extremely large (more than a million records per day in many applications). Our approach is very efficient. We show that our approach requires O(N) in space and time to discover all possible patterns in N events. We discuss a practical implementation of the algorithm in the context of the logs from a large cloud computing system.