Adaptive resonance theory (ART)
The handbook of brain theory and neural networks
Self-Organizing Maps
Discovery of Web Robot Sessions Based on their Navigational Patterns
Data Mining and Knowledge Discovery
Automated Personalisation of Internet Users Using Self-Organising Maps
IDEAL '02 Proceedings of the Third International Conference on Intelligent Data Engineering and Automated Learning
Securing web service by automatic robot detection
ATEC '06 Proceedings of the annual conference on USENIX '06 Annual Technical Conference
Two-level Clustering of Web Sites Using Self-Organizing Maps
Neural Processing Letters
Web robot detection: A probabilistic reasoning approach
Computer Networks: The International Journal of Computer and Telecommunications Networking
Monitoring the application-layer DDoS attacks for popular websites
IEEE/ACM Transactions on Networking (TON)
Modeling human behavior for defense against flash-crowd attacks
ICC'09 Proceedings of the 2009 IEEE international conference on Communications
Web Spambot Detection Based on Web Navigation Behaviour
AINA '10 Proceedings of the 2010 24th IEEE International Conference on Advanced Information Networking and Applications
Web robot detection techniques: overview and limitations
Data Mining and Knowledge Discovery
Feature evaluation for web crawler detection with data mining techniques
Expert Systems with Applications: An International Journal
IEEE Transactions on Neural Networks
Detection of HTTP-GET attack with clustering and information theoretic measurements
FPS'12 Proceedings of the 5th international conference on Foundations and Practice of Security
An extensive study of Web robots traffic
Proceedings of International Conference on Information Integration and Web-based Applications & Services
| Hi-index | 0.00 |
Distributed denials of service (DDoS) attacks are recognized as one of the most damaging attacks on the Internet security today. Recently, malicious web crawlers have been used to execute automated DDoS attacks on web sites across the WWW. In this study, we examine the use of two unsupervised neural network (NN) learning algorithms for the purpose web-log analysis: the Self-Organizing Map (SOM) and Modified Adaptive Resonance Theory 2 (Modified ART2). In particular, through the use of SOM and modified ART2, our work aims to obtain a better insight into the types and distribution of visitors to a public web-site based on their browsing behavior, as well as to investigate the relative differences and/or similarities between malicious web crawlers and other non-malicious visitor groups. The results of our study show that, even though there is a pretty clear separation between malicious web-crawlers and other visitor groups, 52% of malicious crawlers exhibit very 'human-like' browsing behavior and as such pose a particular challenge for future web-site security systems. Also, we show that some of the feature values of malicious crawlers that exhibit very 'human-like' browsing behavior are not significantly different than the features values of human visitors. Additionally, we show that Google, MSN and Yahoo crawlers exhibit distinct crawling behavior.