Efficient Intrusion Detection Based on Static Analysis and Stack Walks
IWSEC '09 Proceedings of the 4th International Workshop on Security: Advances in Information and Computer Security
| Hi-index | 0.00 |
In this dissertation, we describe how we develop novel approaches for host-based anomaly detection. We investigate new ways to improve detection capability without sacrificing false positive performance and efficiency, and present new methods using both dynamic monitoring and static analysis techniques. Most former work used fixed-length subsequences within the system call traces. We propose a novel variable-length pattern extraction algorithm, called LookN, based on loss-less compression techniques. This algorithm is applied on system call traces for anomaly detection purposes. It is computationally simple and efficient. The call stack of program execution can be a very good information source for intrusion detection. There was no prior work on dynamically extracting information from call stack and effectively using it to detect exploits. We propose another new method that we call Vt-Path to do anomaly detection using call stack information. The basic idea is to extract return addresses from the call stack, and generate abstract execution path between two program execution points. Experiments show that our method can detect some attacks that cannot be detected by other approaches, while its convergence and false positive performance is comparable to or better than the other approaches. Models constructed using static analysis have the highly desirable feature that they do not produce false alarms; however, they may still miss attacks. Prior work has shown a trade-off between efficiency and precision. In particular, the more accurate models based upon pushdown automata (PDA) are very inefficient to operate due to non-determinism in stack activity. We present techniques for determinizing PDA models. We provide a formal analysis framework of PDA models and introduce the concepts of determinism and stack-determinism. We then present the VPStatic model, which achieves determinism by extracting information about stack activity of the program. Our results shows that reasonable efficiency needs not be sacrificed for model precision, and deterministic PDA are more efficient to operate than stack-deterministic PDA. In summary, we study different ways to improve intrusion detection system performance. We explore different information sources, different model generating approaches, and different ways of using the information. Several new approaches are proposed.