Analyzing memory accesses in obfuscated x86 executables

Authors:
Michael Venable;Mohamed R. Chouchane;Md Enamul Karim;Arun Lakhotia
Affiliations:
Center for Advanced Computer Studies, University of Louisiana at Lafayette, LA;Center for Advanced Computer Studies, University of Louisiana at Lafayette, LA;Center for Advanced Computer Studies, University of Louisiana at Lafayette, LA;Center for Advanced Computer Studies, University of Louisiana at Lafayette, LA
Venue:
DIMVA'05 Proceedings of the Second international conference on Detection of Intrusions and Malware, and Vulnerability Assessment
Year:
2005

Citing 2
Cited 5

Obfuscation of executable code to improve resistance to static disassembly

Proceedings of the 10th ACM conference on Computer and communications security
Abstracting Stack to Detect Obfuscated Calls in Binaries

SCAM '04 Proceedings of the Source Code Analysis and Manipulation, Fourth IEEE International Workshop

A Method for Detecting Obfuscated Calls in Malicious Binaries

IEEE Transactions on Software Engineering
AWE: improving software analysis through modular integration of static and dynamic analyses

PASTE '07 Proceedings of the 7th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering
Context-sensitive analysis of obfuscated x86 executables

Proceedings of the 2010 ACM SIGPLAN workshop on Partial evaluation and program manipulation
Context-sensitive analysis without calling-context

Higher-Order and Symbolic Computation
Network–Level polymorphic shellcode detection using emulation

DIMVA'06 Proceedings of the Third international conference on Detection of Intrusions and Malware & Vulnerability Assessment

Quantified Score

Hi-index	0.00

Visualization

Abstract

Programmers obfuscate their code to defeat manual or automated analysis. Obfuscations are often used to hide malicious behavior. In particular, malicious programs employ obfuscations of stack-based instructions, such as call and return instructions, to prevent an analyzer from determining which system functions it calls. Instead of using these instructions directly, a combination of other instructions, such as PUSH and POP, are used to achieve the same semantics. This paper presents an abstract interpretation based analysis to detect obfuscation of stack instructions. The approach combines Reps and Balakrishnan's value set analysis (VSA) and Lakhotia and Kumar's Abstract Stack Graph, to create an analyzer that can track stack manipulations where the stack pointer may be saved and restored in memory or registers. The analysis technique may be used to determine obfuscated calls made by a program, an important first step in detecting malicious behavior.