Applying data mining to intrusion detection: the quest for automation, efficiency, and credibility
ACM SIGKDD Explorations Newsletter
Dynamic syslog mining for network failure monitoring
Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining
What Supercomputers Say: A Study of Five System Logs
DSN '07 Proceedings of the 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks
Bad Words: Finding Faults in Spirit's Syslogs
CCGRID '08 Proceedings of the 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid
Failure Prediction in IBM BlueGene/L Event Logs
ICDM '07 Proceedings of the 2007 Seventh IEEE International Conference on Data Mining
Detecting large-scale system problems by mining console logs
Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles
Accurate fault prediction of BlueGene/P RAS logs via geometric reduction
DSNW '10 Proceedings of the 2010 International Conference on Dependable Systems and Networks Workshops (DSN-W)
Framework for enabling system understanding
Euro-Par'11 Proceedings of the 2011 international conference on Parallel Processing - Volume 2
DevOps patterns to scale web applications using cloud services
Proceedings of the 2013 companion publication for conference on Systems, programming, & applications: software for humanity
ELVIS: Extensible Log VISualization
Proceedings of the Tenth Workshop on Visualization for Cyber Security
| Hi-index | 0.00 |
Supercomputers are composed of many diverse components, operated at a variety of scales, and function as a coherent whole. The resulting logs are thus diverse in format, interrelated at multiple scales, and provide evidence of faults across subsystems. When combined with system configuration information, insights on both the downstream effects and upstream causes of events can be determined. However, difficulties in joining the data and expressing complex queries slow the speed at which actionable insights can be obtained. Effectively connecting data experts and data miners faces similar hurdles. This paper describes our experience with applying the Splunk log analysis tool as a vehicle to combine both data, and people. Splunk's search language, lookups, macros, and subsearches reduce hours of tedium to seconds of simplicity, and its tags, saved searches, and dashboards offer both operational insights and collaborative vehicles.