Learning automata: an introduction
Learning automata: an introduction
Optimal crawling strategies for web search engines
Proceedings of the 11th international conference on World Wide Web
Monitoring the dynamic web to respond to continuous queries
WWW '03 Proceedings of the 12th international conference on World Wide Web
IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
| Hi-index | 0.00 |
We consider the problem of polling web pages as a strategy for monitoring the world wide web. The problem consists of repeatedly polling a selection of web pages so that changes that occur over time are detected. In particular, we consider the case where we are constrained to poll a maximumnumber of web pages per unit of time. Thus, the issue at stake is one of determining which web pages are to be polled, and we attempt to do it in a manner that maximizes the number of changes detected. We solve the problem by first modelling it as a Stochastic Non-linear Fractional Knapsack Problem. We then present a completely new on-line Learning Automata (LA) system, namely, the Hierarchy of Twofold Resource Allocation Automata(H-TRAA), whose primitive component is a Twofold Resource Allocation Automaton(TRAA). Both the TRAA and the H-TRAA have been proven to be asymptotically optimal. Finally, we demonstrate empirically that H-TRAA provides orders of magnitudefaster convergence compared to the LAKG which represents the state-of-the-art. Further, in contrast to the LAKG, H-TRAA scales sub-linearly. Based on these results, we believe that the H-TRAA has a tremendous potential to handle demanding real-world applications, particularly those which deal with the world wide web.