Maintaining views incrementally
SIGMOD '93 Proceedings of the 1993 ACM SIGMOD international conference on Management of data
Temporal View Self-Maintenance
EDBT '00 Proceedings of the 7th International Conference on Extending Database Technology: Advances in Database Technology
Selection of Views to Materialize in a Data Warehouse
ICDT '97 Proceedings of the 6th International Conference on Database Theory
On the Discovery of Interesting Patterns in Association Rules
VLDB '98 Proceedings of the 24rd International Conference on Very Large Data Bases
Publish/Subscribe on the Web at Extreme Speed
VLDB '00 Proceedings of the 26th International Conference on Very Large Data Bases
Automated Selection of Materialized Views and Indexes in SQL Databases
VLDB '00 Proceedings of the 26th International Conference on Very Large Data Bases
SCRIBE: The Design of a Large-Scale Event Notification Infrastructure
NGC '01 Proceedings of the Third International COST264 Workshop on Networked Group Communication
Answering queries using views: A survey
The VLDB Journal — The International Journal on Very Large Data Bases
Cayuga: a high-performance event processing engine
Proceedings of the 2007 ACM SIGMOD international conference on Management of data
Statistical properties of community structure in large social and information networks
Proceedings of the 17th international conference on World Wide Web
PNUTS: Yahoo!'s hosted data serving platform
Proceedings of the VLDB Endowment
Feeding frenzy: selectively materializing users' event feeds
Proceedings of the 2010 ACM SIGMOD International Conference on Management of data
The little engine(s) that could: scaling online social networks
Proceedings of the ACM SIGCOMM 2010 conference
Traffic analysis of a Web proxy caching hierarchy
IEEE Network: The Magazine of Global Internetworking
Poor man's social network: consistently trade freshness for scalability
WebApps'12 Proceedings of the 3rd USENIX conference on Web Application Development
Challenges in enabling social application at scale: cloudDB'12 invited-keynote talk abstract
Proceedings of the fourth international workshop on Cloud data management
Archiving the relaxed consistency web
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Supporting distributed feed-following apps over edge devices
Proceedings of the VLDB Endowment
Hi-index | 0.00 |
Internet users spend billions of minutes per month on sites like Facebook and Twitter. These sites support feed following, where users "follow" activity streams associated with other users and entities. Followers get personalized feeds that blend streams produced by those followed. The emphasis on recency and relevance, and the highly variable fan-out of the follows graph, make this feature difficult to implement at the scale seen in major social networks. In this paper, we place feed following in the context of existing research areas and highlight the novel data management challenges that it poses, with the goal of stimulating research in this new direction. We discuss solutions based on pub/sub, caching, and materialized views, and argue that none of these existing approaches fully exploit the unique characteristics of feed following. The number of distinct queries and the query rate per second that a feed following system must support are huge, but queries have simple structure and overlap. The system must handle high throughput input streams, but results are heavily biased toward recent events. The number of users is large, but they exhibit diurnal behavior, and we can dynamically modify the system to optimize for currently active users. These characteristics offer many opportunities for optimization, and the potential gains are substantial.