Distinguishing humans from robots in web search logs: preliminary results using query rates and intervals

Authors:
Omer Duskin;Dror G. Feitelson
Affiliations:
The Hebrew University of Jerusalem, Jerusalem, Israel;The Hebrew University of Jerusalem, Jerusalem, Israel
Venue:
Proceedings of the 2009 workshop on Web Search Click Data
Year:
2009

Citing 6
Cited 7

How are we searching the world wide web?: a comparison of nine search engine transaction logs

Information Processing and Management: an International Journal - Special issue: Formal methods for information retrieval
Automated gathering of Web information: An in-depth examination of agents interacting with search engines

ACM Transactions on Internet Technology (TOIT)
Web Search: Public Searching of the Web (Information Science and Knowledge Management)

Web Search: Public Searching of the Web (Information Science and Knowledge Management)
Sliding window technique for the web log analysis

Proceedings of the 16th international conference on World Wide Web
Web robot detection: A probabilistic reasoning approach

Computer Networks: The International Journal of Computer and Telecommunications Networking
Evaluation of web robot discovery techniques: a benchmarking study

ICDM'06 Proceedings of the 6th Industrial Conference on Data Mining conference on Advances in Data Mining: applications in Medicine, Web Mining, Marketing, Image and Signal Mining

A brief survey on sequence classification

ACM SIGKDD Explorations Newsletter
Web robot detection techniques: overview and limitations

Data Mining and Knowledge Discovery
Adversarial Web Search

Foundations and Trends in Information Retrieval
Web robot detection based on pattern-matching technique

Journal of Information Science
Analysis of web logs: challenges and findings

PERFORM'10 Proceedings of the 2010 IFIP WG 6.3/7.3 international conference on Performance Evaluation of Computer and Communication Systems: milestones and future challenges
Sentiment classification with supervised sequence embedding

ECML PKDD'12 Proceedings of the 2012 European conference on Machine Learning and Knowledge Discovery in Databases - Volume Part I
Identifying user sessions from web server logs with integer programming

Intelligent Data Analysis - Business Analytics and Intelligent Optimization

Quantified Score

Hi-index	0.00

Visualization

Abstract

The workload on web search engines is actually multiclass, being derived from the activities of both human users and automated robots. It is important to distinguish between these two classes in order to reliably characterize human web search behavior, and to study the effect of robot activity. We suggest an approach based on a multi-dimensional characterization of search sessions, and take first steps towards implementing it by studying the interaction between the query submittal rate and the minimal interval of time between different queries.