Efficient crawling through URL ordering
WWW7 Proceedings of the seventh international conference on World Wide Web 7
Estimating frequency of change
ACM Transactions on Internet Technology (TOIT)
Scheduling Algorithms for Web Crawling
LA-WEBMEDIA '04 Proceedings of the WebMedia & LA-Web 2004 Joint Conference 10th Brazilian Symposium on Multimedia and the Web 2nd Latin American Web Congress
Parallel crawling for online social networks
Proceedings of the 16th international conference on World Wide Web
Effective change detection using sampling
VLDB '02 Proceedings of the 28th international conference on Very Large Data Bases
RankMass crawler: a crawler with high personalized pagerank coverage guarantee
VLDB '07 Proceedings of the 33rd international conference on Very large data bases
Recrawl scheduling based on information longevity
Proceedings of the 17th international conference on World Wide Web
Design trade-offs for search engine caching
ACM Transactions on the Web (TWEB)
Social Network - An Autonomous System Designed for Radio Recommendation
CASON '09 Proceedings of the 2009 International Conference on Computational Aspects of Social Networks
SHARC: framework for quality-conscious web archiving
Proceedings of the VLDB Endowment
Earthquake shakes Twitter users: real-time event detection by social sensors
Proceedings of the 19th international conference on World wide web
Mining Twitter in the Cloud: A Case Study
CLOUD '10 Proceedings of the 2010 IEEE 3rd International Conference on Cloud Computing
Proceedings of the 17th ACM SIGKDD International Conference Tutorials
Twitter catches the flu: detecting influenza epidemics using Twitter
EMNLP '11 Proceedings of the Conference on Empirical Methods in Natural Language Processing
TwitterEcho: a distributed focused crawler to support open research with twitter data
Proceedings of the 21st international conference companion on World Wide Web
Automated Twitter data collecting tool for data mining in social network
Proceedings of the 2012 ACM Research in Applied Computation Symposium
Hi-index | 0.00 |
Social network preserves the life of users and provides great potential for journalists, sociologists and business analysts. Crawling data from social network is a basic step for social network information analysis and processing. As the network becomes huge and information on the network updates faster than web pages, crawling is more difficult because of the limitations of bandwidth, politeness etiquette and computation power. To extract fresh information from social network efficiently and effectively, this paper presents a novel crawling method of social network. To discover the feature of social network, we gather data from real social network, analyze them and build a model to describe the discipline of users' behavior. With the modeled behavior, we propose methods to predict users' behavior. According to the prediction, we schedule our crawler more reasonably and extract more fresh information. Experimental results demonstrate that our strategies could obtain information from SNS efficiently and effectively.