WhatNext: A Prediction System for Web Requests using N-gram Sequence Models

  • Authors:
  • Zhong Su;Qiang Yang;Ye Lu;Hongjiang Zhang

  • Affiliations:
  • -;-;-;-

  • Venue:
  • WISE '00 Proceedings of the First International Conference on Web Information Systems Engineering (WISE'00)-Volume 1 - Volume 1
  • Year:
  • 2000

Quantified Score

Hi-index 0.00

Visualization

Abstract

As an increasing number of users access information on the web, there is a great opportunity to learn from the server logs to learn about the users' probable actions in the future. In this paper, we present an n-gram based model to utilize path profiles of users from very large data sets to predict the users' future requests. Since this is a prediction system, we cannot measure the recall in a traditional sense. We, therefore, present the notion of applicability to give a measure of the ability to predict the next document. Our model is based on a simple extension of existing point-based models for such predictions, but our result show for n-gram based prediction when n is greater than three are, we can increase precision by 20% or more for two realistic web logs. In addition, we present an efficient method that can compress our model to 30% of its original size so that the model can be loaded in main memory. Our result can potentially be applied to a wide range of applications on the web, including pre-sending, prefetching, enhancement of recommendation systems as well as web caching policies. Our tests are based on three realistic web logs. Our algorithm is implemented in a prediction system called WhatNext, which shows a marked improvement in precision and applicability over previous approaches.