Real-world polymorphic attack detection using network-level emulation

• Authors: Michalis Polychronakis;Kostas G. Anagnostakis;Evangelos P. Markatos
• Venue: Proceedings of the 4th annual workshop on Cyber security and information intelligence research: developing strategies to meet the cyber security and information intelligence challenges ahead
• Year: 2008

As state-of-the-art attack detection technology becomes more prevalent, attackers have started to employ techniques such as code obfuscation and polymorphism to defeat these defenses. We have recently proposed network-level emulation, a heuristic detection method that scans network traffic to detect polymorphic attacks. Our approach uses a CPU emulator to dynamically analyze every potential instruction sequence in the inspected traffic, aiming to identify the execution behavior of certain malicious code classes, such as self-decrypting polymorphic shellcode. Network-level emulation does not rely on any exploit or vulnerability specific signatures, which allows the detection of previously unknown attacks, while the actual execution of the attack code makes the detector robust to evasion techniques such as self-modifying code. After more than a year of continuous operation in production networks, our prototype implementation has captured more than a million attacks against real systems, employing a highly diverse set of exploits, often against less widely used vulnerable services, while so far has not resulted to any false positives.