View maintenance in a warehousing environment
SIGMOD '95 Proceedings of the 1995 ACM SIGMOD international conference on Management of data
Implementing data cubes efficiently
SIGMOD '96 Proceedings of the 1996 ACM SIGMOD international conference on Management of data
The anatomy of a large-scale hypertextual Web search engine
WWW7 Proceedings of the seventh international conference on World Wide Web 7
Efficient crawling through URL ordering
WWW7 Proceedings of the seventh international conference on World Wide Web 7
Focused crawling: a new approach to topic-specific Web resource discovery
WWW '99 Proceedings of the eighth international conference on World Wide Web
An adaptive model for optimizing performance of an incremental web crawler
Proceedings of the 10th international conference on World Wide Web
ACM Transactions on Internet Technology (TOIT)
Managing periodically updated data in relational databases: a stochastic modeling approach
Journal of the ACM (JACM)
Proceedings of the 11th international conference on World Wide Web
Optimal crawling strategies for web search engines
Proceedings of the 11th international conference on World Wide Web
Best-effort cache synchronization with source cooperation
Proceedings of the 2002 ACM SIGMOD international conference on Management of data
Supporting diverse mobile applications with client profiles
WOWMOM '02 Proceedings of the 5th ACM international workshop on Wireless mobile multimedia
Engineering web cache consistency
ACM Transactions on Internet Technology (TOIT)
Text Retrieval Systems for the Web
Programming and Computing Software
Minimizing View Sets without Losing Query-Answering Power
ICDT '01 Proceedings of the 8th International Conference on Database Theory
The Evolution of the Web and Implications for an Incremental Crawler
VLDB '00 Proceedings of the 26th International Conference on Very Large Data Bases
Proceedings of the 27th International Conference on Very Large Data Bases
Update Propagation Strategies for Improving the Quality of Data on the Web
Proceedings of the 27th International Conference on Very Large Data Bases
Profile-Based Data Delivery for Web Applications
EDBT '02 Proceedings of the Worshops XMLDM, MDDE, and YRWS on XML-Based Data Management and Multimedia Engineering-Revised Papers
A First Experience in Archiving the French Web
ECDL '02 Proceedings of the 6th European Conference on Research and Advanced Technology for Digital Libraries
Agents, Crawlers, and Web Retrieval
CIA '02 Proceedings of the 6th International Workshop on Cooperative Information Agents VI
Adaptive on-line page importance computation
WWW '03 Proceedings of the 12th international conference on World Wide Web
Monitoring the dynamic web to respond to continuous queries
WWW '03 Proceedings of the 12th international conference on World Wide Web
On staleness and the delivery of web pages
CASCON '01 Proceedings of the 2001 conference of the Centre for Advanced Studies on Collaborative research
Repository synchronization in the OAI framework
Proceedings of the 3rd ACM/IEEE-CS joint conference on Digital libraries
TCP Nice: a mechanism for background transfers
ACM SIGOPS Operating Systems Review - OSDI '02: Proceedings of the 5th symposium on Operating systems design and implementation
Estimating frequency of change
ACM Transactions on Internet Technology (TOIT)
ICDE '04 Proceedings of the 20th International Conference on Data Engineering
What's new on the web?: the evolution of the web from a search engine perspective
Proceedings of the 13th international conference on World Wide Web
A framework for analysis of data freshness
Proceedings of the 2004 international workshop on Information quality in information systems
Multi-Agent Patrolling with Reinforcement Learning
AAMAS '04 Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems - Volume 3
The Evolution of Link-Attributes for Pages and Its Implications on Web Crawling
WI '04 Proceedings of the 2004 IEEE/WIC/ACM International Conference on Web Intelligence
A Weighted Freshness Metric for Maintaining Search Engine Local Repository
WI '04 Proceedings of the 2004 IEEE/WIC/ACM International Conference on Web Intelligence
High performance crawling system
Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval
Scheduling Queries to Improve the Freshness of a Website
World Wide Web
Measuring cache freshness by additive age
ACM SIGOPS Operating Systems Review
On demand synchronization and load distribution for database grid-based web applications
Data & Knowledge Engineering
Modeling and Managing Content Changes in Text Databases
ICDE '05 Proceedings of the 21st International Conference on Data Engineering
ACM Computing Surveys (CSUR)
TCP Nice: a mechanism for background transfers
OSDI '02 Proceedings of the 5th symposium on Operating systems design and implementationCopyright restrictions prevent ACM from being able to make the PDFs for this conference available for downloading
WWW '05 Proceedings of the 14th international conference on World Wide Web
The infocious web search engine: improving web searching through linguistic analysis
WWW '05 Special interest tracks and posters of the 14th international conference on World Wide Web
Crawling a country: better strategies than breadth-first for web page ordering
WWW '05 Special interest tracks and posters of the 14th international conference on World Wide Web
Trend detection through temporal link analysis
Journal of the American Society for Information Science and Technology - Special issue: Webometrics
Adaptive pull-based policies for wide area data delivery
ACM Transactions on Database Systems (TODS)
Web dynamics and their ramifications for the development of web search engines
Computer Networks: The International Journal of Computer and Telecommunications Networking - Web dynamics
Stanford WebBase components and applications
ACM Transactions on Internet Technology (TOIT)
Evaluation of crawling policies for a web-repository crawler
Proceedings of the seventeenth conference on Hypertext and hypermedia
The discoverability of the web
Proceedings of the 16th international conference on World Wide Web
Design and evaluation of a continuous consistency model for replicated services
OSDI'00 Proceedings of the 4th conference on Symposium on Operating System Design & Implementation - Volume 4
NPS: a non-interfering deployable web perfectching system
USITS'03 Proceedings of the 4th conference on USENIX Symposium on Internet Technologies and Systems - Volume 4
Efficient Monitoring Algorithm for Fast News Alerts
IEEE Transactions on Knowledge and Data Engineering
Modeling and managing changes in text databases
ACM Transactions on Database Systems (TODS)
Effective change detection using sampling
VLDB '02 Proceedings of the 28th international conference on Very Large Data Bases
Using latency-recency profiles for data delivery on the web
VLDB '02 Proceedings of the 28th international conference on Very Large Data Bases
Stochastic consistency, and scalable pull-based caching for erratic data stream sources
VLDB '04 Proceedings of the Thirtieth international conference on Very large data bases - Volume 30
WIC: a general-purpose algorithm for monitoring web information sources
VLDB '04 Proceedings of the Thirtieth international conference on Very large data bases - Volume 30
Designing clustering-based web crawling policies for search engine crawlers
Proceedings of the sixteenth ACM conference on Conference on information and knowledge management
Crawl ordering by search impact
WSDM '08 Proceedings of the 2008 International Conference on Web Search and Data Mining
Selective 2-versioning for concurrency control in data warehousing systems: S2V
International Journal of Computer Applications in Technology
A new aggregation policy for RSS services
Proceedings of the 2008 international workshop on Context enabled source and service selection, integration and adaptation: organized with the 17th International World Wide Web Conference (WWW 2008)
Data & Knowledge Engineering
Maintaining dynamic channel profiles on the web
Proceedings of the VLDB Endowment
Parallel crawler architecture and web page change detection
WSEAS Transactions on Computers
On the feasibility of geographically distributed web crawling
Proceedings of the 3rd international conference on Scalable information systems
Sitemaps: above and beyond the crawl of duty
Proceedings of the 18th international conference on World wide web
User-centric content freshness metrics for search engines
Proceedings of the 18th international conference on World wide web
Web spam filtering in internet archives
Proceedings of the 5th International Workshop on Adversarial Information Retrieval on the Web
Scheduling to minimize staleness and stretch in real-time data warehouses
Proceedings of the twenty-first annual symposium on Parallelism in algorithms and architectures
The gardener's problem for web information monitoring
Proceedings of the 18th ACM conference on Information and knowledge management
A capture-recapture sampling standardization for improving Internet meta-search
Computer Standards & Interfaces
Non-functional data collection for adaptive business processes and decision making
Proceedings of the 4th International Workshop on Middleware for Service Oriented Computing
Truth discovery and copying detection in a dynamic world
Proceedings of the VLDB Endowment
SHARC: framework for quality-conscious web archiving
Proceedings of the VLDB Endowment
Adaptive immune system inspired perimeter patrol control strategy
IROS'09 Proceedings of the 2009 IEEE/RSJ international conference on Intelligent robots and systems
Computer Networks: The International Journal of Computer and Telecommunications Networking
Caching and Materialization for Web Databases
Foundations and Trends in Databases
Implementation of a web robot and statistics on the Korean web
HSI'03 Proceedings of the 2nd international conference on Human.society@internet
Multi-agent patrolling: an empirical analysis of alternative architectures
MABS'02 Proceedings of the 3rd international conference on Multi-agent-based simulation II
The adaptive web
Efficiently detecting webpage updates using samples
ICWE'07 Proceedings of the 7th international conference on Web engineering
Optimizing content freshness of relations extracted from the web using keyword search
Proceedings of the 2010 ACM SIGMOD International Conference on Management of data
On trade-offs in event delivery systems
Proceedings of the Fourth ACM International Conference on Distributed Event-Based Systems
Influence of different execution models on patrolling ant behaviors: from agents to robots
Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 3 - Volume 3
Clustering-based incremental web crawling
ACM Transactions on Information Systems (TOIS)
Tuning QoD in stream processing engines
ADC '10 Proceedings of the Twenty-First Australasian Conference on Database Technologies - Volume 104
Towards a quality-oriented real-time web crawler
WISM'10 Proceedings of the 2010 international conference on Web information systems and mining
The SHARC framework for data quality in Web archiving
The VLDB Journal — The International Journal on Very Large Data Bases
Best-effort refresh strategies for content-based RSS feed aggregation
WISE'10 Proceedings of the 11th international conference on Web information systems engineering
Caché: caching location-enhanced content to improve user privacy
MobiSys '11 Proceedings of the 9th international conference on Mobile systems, applications, and services
Journal of Web Engineering
A framework for incremental deep web crawler based on URL classification
WISM'11 Proceedings of the 2011 international conference on Web information systems and mining - Volume Part II
UpStream: storage-centric load management for streaming applications with update semantics
The VLDB Journal — The International Journal on Very Large Data Bases
An empirical study on the change of web pages
APWeb'05 Proceedings of the 7th Asia-Pacific web conference on Web Technologies Research and Development
A precise metric for measuring how much web pages change
DASFAA'06 Proceedings of the 11th international conference on Database Systems for Advanced Applications
State transfer graph: an efficient tool for webview maintenance
WAIM'05 Proceedings of the 6th international conference on Advances in Web-Age Information Management
Competitive freshness algorithms for wait-free data objects
Euro-Par'06 Proceedings of the 12th international conference on Parallel Processing
An ontology-guided approach to change detection of the semantic web data
Journal on Data Semantics V
A hybrid approach for refreshing web page repositories
DASFAA'05 Proceedings of the 10th international conference on Database Systems for Advanced Applications
The space complexity of transactional interactive reads
Proceedings of the 1st International Workshop on Hot Topics in Cloud Data Processing
A hierarchical document clustering environment based on the induced bisecting k-means
FQAS'06 Proceedings of the 7th international conference on Flexible Query Answering Systems
Semi-automatic creation and maintenance of web resources with webtopic
EWMF'05/KDO'05 Proceedings of the 2005 joint international conference on Semantics, Web and Mining
ICWE'12 Proceedings of the 12th international conference on Web Engineering
Towards benchmarking stream data warehouses
Proceedings of the fifteenth international workshop on Data warehousing and OLAP
A Hybrid Approach for Web Change Detection
International Journal of Information Technology and Web Engineering
Hi-index | 0.00 |
In this paper we study how to refresh a local copy of an autonomous data source to maintain the copy up-to-date. As the size of the data grows, it becomes more difficult to maintain the copy \ fresh, “making it crucial to synchronize the copy effectively. We define two freshness metrics, change models of the underlying data, and synchronization policies. We analytically study how effective the various policies are. We also experimentally verify our analysis, based on data collected from 270 web sites for more than 4 months, and we show that our new policy improves the \ freshness” very significantly compared to current policies in use.