A polymorphic shellcode detection mechanism in the network

Authors:
Hsiang-Lun Huang;Tzong-Jye Liu;Kuong-Ho Chen;Chyi-Ren Dow;Lih-Chyau Wuu
Affiliations:
Feng Chia University, Taichung, Taiwan, R.O.C.;Feng Chia University, Taichung, Taiwan, R.O.C.;Feng Chia University, Taichung, Taiwan, R.O.C.;Feng Chia University, Taichung, Taiwan, R.O.C.;National Yunlin University of Science and Technology, Yunlin, Taiwan, R.O.C.
Venue:
Proceedings of the 2nd international conference on Scalable information systems
Year:
2007

Citing 8
Cited 1

RAD: A Compile-Time Solution to Buffer Overflow Attacks

ICDCS '01 Proceedings of the The 21st International Conference on Distributed Computing Systems
A Framework for Detecting Network-based Code Injection Attacks Targeting Windows and UNIX

ACSAC '05 Proceedings of the 21st Annual Computer Security Applications Conference
SmashGuard: A Hardware Solution to Prevent Security Attacks on the Function Return Address

IEEE Transactions on Computers
PointguardTM: protecting pointers from buffer overflow vulnerabilities

SSYM'03 Proceedings of the 12th conference on USENIX Security Symposium - Volume 12
StackGuard: automatic adaptive detection and prevention of buffer-overflow attacks

SSYM'98 Proceedings of the 7th conference on USENIX Security Symposium - Volume 7
Accurate buffer overflow detection via abstract payload execution

RAID'02 Proceedings of the 5th international conference on Recent advances in intrusion detection
Network–Level polymorphic shellcode detection using emulation

DIMVA'06 Proceedings of the Third international conference on Detection of Intrusions and Malware & Vulnerability Assessment
Experiences using minos as a tool for capturing and analyzing novel worms for unknown vulnerabilities

DIMVA'05 Proceedings of the Second international conference on Detection of Intrusions and Malware, and Vulnerability Assessment

Protecting against address space layout randomisation (ASLR) compromises and return-to-libc attacks using network intrusion detection systems

International Journal of Automation and Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Buffer overflow attack is a major security problem in recent years. The polymorphism technique for shellcode becomes more and more popular along with development of Internet. This paper proposes a method to detect the polymorphic shellcode for Windows operating system. The proposed approach relies on an IA-32 CPU emulator that executes instruction sequences and analyze the behavior of polymorphic shellcode. The experimental results show that the approach is able to detect polymorphic shellcode accurately.