Proceedings of the 9th international World Wide Web conference on Computer networks : the international journal of computer and telecommunications netowrking
Probability and Statistics with Reliability, Queuing and Computer Science Applications
Probability and Statistics with Reliability, Queuing and Computer Science Applications
Estimating frequency of change
ACM Transactions on Internet Technology (TOIT)
Estimation of internet file-access/modification rates from indirect data
ACM Transactions on Modeling and Computer Simulation (TOMACS)
The SHARC framework for data quality in Web archiving
The VLDB Journal — The International Journal on Very Large Data Bases
| Hi-index | 0.00 |
Estimating the rate of Web page updates helps in improving the Web crawler's scheduling policy. But, most of the Web sources are autonomous and updated independently. Clients like Web crawlers are not aware of when and how often the sources change. Unlike other studies, the process of Web page updates is modeled as nonhomogeneous Poisson process and focus on determining localized rate of updates. Then various rate estimators are discussed, showing experimentally how precise they are. This paper explores two classes of problems. Firstly the localized rate of updates is estimated by dividing the given sequence of independent and inconsistent update points into consistent windows. From various experimental comparisons, the proposed Weibull estimator outperforms Duane plot(another proposed estimator) and other estimators proposed by Cho et al. and Norman Matloff in 91.5%(90.6%) of the whole windows for synthetic(real Web) datasets. Secondly, the future update points are predicted based on most recent window and it is found that Weibull estimator has higher precision compared to other estimators.