Behavioral clustering of HTTP-based malware and signature generation using malicious network traces

Authors:
Roberto Perdisci;Wenke Lee;Nick Feamster
Affiliations:
College of Computing, Georgia Institute of Technology, Atlanta, GA and Damballa, Inc., Atlanta, GA;College of Computing, Georgia Institute of Technology, Atlanta, GA;College of Computing, Georgia Institute of Technology, Atlanta, GA
Venue:
NSDI'10 Proceedings of the 7th USENIX conference on Networked systems design and implementation
Year:
2010

Citing 18
Cited 32

Algorithms for clustering data

Algorithms for clustering data
Data clustering: a review

ACM Computing Surveys (CSUR)
On Clustering Validation Techniques

Journal of Intelligent Information Systems
X-means: Extending K-means with Efficient Estimation of the Number of Clusters

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Polygraph: Automatically Generating Signatures for Polymorphic Worms

SP '05 Proceedings of the 2005 IEEE Symposium on Security and Privacy
MisleadingWorm Signature Generators Using Deliberate Noise Injection

SP '06 Proceedings of the 2006 IEEE Symposium on Security and Privacy
Hamsa: Fast Signature Generation for Zero-day PolymorphicWorms with Provable Attack Resilience

SP '06 Proceedings of the 2006 IEEE Symposium on Security and Privacy
Automated worm fingerprinting

OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
An architecture for generating semantics-aware signatures

SSYM'05 Proceedings of the 14th conference on USENIX Security Symposium - Volume 14
Behavior-based spyware detection

USENIX-SS'06 Proceedings of the 15th conference on USENIX Security Symposium - Volume 15
Dytan: a generic dynamic taint analysis framework

Proceedings of the 2007 international symposium on Software testing and analysis
Panorama: capturing system-wide information flow for malware detection and analysis

Proceedings of the 14th ACM conference on Computer and communications security
Spamming botnets: signatures and characteristics

Proceedings of the ACM SIGCOMM 2008 conference on Data communication
CloudAV: N-version antivirus in the network cloud

SS'08 Proceedings of the 17th conference on Security symposium
BotMiner: clustering analysis of network traffic for protocol- and structure-independent botnet detection

SS'08 Proceedings of the 17th conference on Security symposium
Studying spamming botnets using Botlab

NSDI'09 Proceedings of the 6th USENIX symposium on Networked systems design and implementation
Automated classification and analysis of internet malware

RAID'07 Proceedings of the 10th international conference on Recent advances in intrusion detection
Paragraph: thwarting signature learning by training maliciously

RAID'06 Proceedings of the 9th international conference on Recent Advances in Intrusion Detection

Outsourcing home network security

Proceedings of the 2010 ACM SIGCOMM workshop on Home networks
Instrumenting home networks

Proceedings of the 2010 ACM SIGCOMM workshop on Home networks
On challenges in evaluating malware clustering

RAID'10 Proceedings of the 13th international conference on Recent advances in intrusion detection
Instrumenting home networks

ACM SIGCOMM Computer Communication Review
Building a dynamic reputation system for DNS

USENIX Security'10 Proceedings of the 19th USENIX conference on Security
ARROW: GenerAting SignatuRes to Detect DRive-By DOWnloads

Proceedings of the 20th international conference on World wide web
Sandnet: network traffic analysis of malicious software

Proceedings of the First Workshop on Building Analysis Datasets and Gathering Experience Returns for Security
Detecting bots via incremental LS-SVM learning with dynamic feature adaptation

Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining
Measuring pay-per-install: the commoditization of malware distribution

SEC'11 Proceedings of the 20th USENIX conference on Security
JACKSTRAWS: picking command and control connections from bot traffic

SEC'11 Proceedings of the 20th USENIX conference on Security
Machine learning in computer forensics (and the lessons learned from machine learning in computer security)

Proceedings of the 4th ACM workshop on Security and artificial intelligence
BitShred: feature hashing malware for scalable triage and semantic analysis

Proceedings of the 18th ACM conference on Computer and communications security
Research challenges towards the Future Internet

Computer Communications
Understanding the prevalence and use of alternative plans in malware with network games

Proceedings of the 27th Annual Computer Security Applications Conference
FORECAST: skimming off the malware cream

Proceedings of the 27th Annual Computer Security Applications Conference
Detecting malware's failover C&C strategies with squeeze

Proceedings of the 27th Annual Computer Security Applications Conference
Detecting environment-sensitive malware

RAID'11 Proceedings of the 14th international conference on Recent Advances in Intrusion Detection
Automatic input rectification

Proceedings of the 34th International Conference on Software Engineering
ReasONets: a fuzzy-based approach for reasoning on network incidents

Proceedings of the 2012 ACM conference on Computer and communications security
BotFinder: finding bots in network traffic without deep packet inspection

Proceedings of the 8th international conference on Emerging networking experiments and technologies
VAMO: towards a fully automated malware clustering validity analysis

Proceedings of the 28th Annual Computer Security Applications Conference
Scalable fine-grained behavioral clustering of HTTP-based malware

Computer Networks: The International Journal of Computer and Telecommunications Networking
Adversarial attacks against intrusion detection systems: Taxonomy, solutions and open issues

Information Sciences: an International Journal
Exploiting visual appearance to cluster and detect rogue software

Proceedings of the 28th Annual ACM Symposium on Applied Computing
Tracking memory writes for malware classification and code reuse identification

DIMVA'12 Proceedings of the 9th international conference on Detection of Intrusions and Malware, and Vulnerability Assessment
NetGator: malware detection using program interactive challenges

DIMVA'12 Proceedings of the 9th international conference on Detection of Intrusions and Malware, and Vulnerability Assessment
PhishLive: a view of phishing and malware attacks from an edge router

PAM'13 Proceedings of the 14th international conference on Passive and Active Measurement
SigMal: a static signal processing based malware triage

Proceedings of the 29th Annual Computer Security Applications Conference
Driving in the cloud: an analysis of drive-by download operations and abuse reporting

DIMVA'13 Proceedings of the 10th international conference on Detection of Intrusions and Malware, and Vulnerability Assessment
ProVeX: detecting botnets with encrypted command and control channels

DIMVA'13 Proceedings of the 10th international conference on Detection of Intrusions and Malware, and Vulnerability Assessment
ExecScent: mining for new C&C domains in live networks with adaptive control protocol templates

SEC'13 Proceedings of the 22nd USENIX conference on Security
Situational awareness through reasoning on network incidents

Proceedings of the 4th ACM conference on Data and application security and privacy

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present a novel network-level behavioral malware clustering system. We focus on analyzing the structural similarities among malicious HTTP traffic traces generated by executing HTTP-based malware. Our work is motivated by the need to provide quality input to algorithms that automatically generate network signatures. Accordingly, we define similarity metrics among HTTP traces and develop our system so that the resulting clusters can yield high-quality malware signatures. We implemented a proof-of-concept version of our network-level malware clustering system and performed experiments with more than 25,000 distinct malware samples. Results from our evaluation, which includes real-world deployment, confirm the effectiveness of the proposed clustering system and show that our approach can aid the process of automatically extracting network signatures for detecting HTTP traffic generated by malware-compromised machines.