Bayesian classification (AutoClass): theory and results
Advances in knowledge discovery and data mining
A hierarchical approach to wrapper induction
Proceedings of the third annual conference on Autonomous Agents
ACM Computing Surveys (CSUR)
A vector space model for automatic indexing
Communications of the ACM
Machine Learning
A brief survey of web data extraction tools
ACM SIGMOD Record
Introduction to Modern Information Retrieval
Introduction to Modern Information Retrieval
RoadRunner: Towards Automatic Data Extraction from Large Web Sites
Proceedings of the 27th International Conference on Very Large Data Bases
Mining data records in Web pages
Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining
Probe, Cluster, and Discover: Focused Extraction of QA-Pagelets from the Deep Web
ICDE '04 Proceedings of the 20th International Conference on Data Engineering
Automatic web news extraction using tree edit distance
Proceedings of the 13th international conference on World Wide Web
Web data extraction based on partial tree alignment
WWW '05 Proceedings of the 14th international conference on World Wide Web
IEEE Transactions on Knowledge and Data Engineering
A Survey of Web Information Extraction Systems
IEEE Transactions on Knowledge and Data Engineering
Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)
Data Extraction from Semi-structured Web Pages by Clustering
WI '06 Proceedings of the 2006 IEEE/WIC/ACM International Conference on Web Intelligence
Extracting lists of data records from semi-structured web pages
Data & Knowledge Engineering
Overview of autofeed: an unsupervised learning system for generating webfeeds
AAAI'06 proceedings of the 21st national conference on Artificial intelligence - Volume 2
Automatic data record detection in web pages
KSEM'07 Proceedings of the 2nd international conference on Knowledge science, engineering and management
Using clustering for web information extraction
AI'07 Proceedings of the 20th Australian joint conference on Advances in artificial intelligence
The lixto project: exploring new frontiers of web data extraction
BNCOD'06 Proceedings of the 23rd British National Conference on Databases, conference on Flexible and Efficient Information Handling
Alignment-based surface patterns for factoid question answering systems
Integrated Computer-Aided Engineering - Selected papers from the IEEE Conference on Information Reuse and Integration (IRI), July 13-15, 2008
Survey of modular ontology techniques and their applications in the biomedical domain
Integrated Computer-Aided Engineering - Selected papers from the IEEE Conference on Information Reuse and Integration (IRI), July 13-15, 2008
A supervised learning approach to biological question answering
Integrated Computer-Aided Engineering - Selected papers from the IEEE Conference on Information Reuse and Integration (IRI), July 13-15, 2008
A practical method for browsing a relational database using a standard search engine
Integrated Computer-Aided Engineering - Selected papers from the IEEE Conference on Information Reuse and Integration (IRI), July 13-15, 2008
Using conditional random fields for result identification in biomedical abstracts
Integrated Computer-Aided Engineering
Ubiquitous web navigation through harvesting embedded semantic data: A mobile scenario
Integrated Computer-Aided Engineering - Anniversary Volume: Celebrating 20 Years of Excellence
A model for mining material properties for radiation shielding
Integrated Computer-Aided Engineering
A novel approach to data deduplication over the engineering-oriented cloud systems
Integrated Computer-Aided Engineering
Community detection for proximity alignment
Integrated Computer-Aided Engineering
| Hi-index | 0.00 |
This paper describes a new approach to the use of clustering for automatic data detection in semi-structured web pages. Unlike most exiting web information extraction approaches that usually apply wrapper induction techniques to manually labelled web pages, this approach avoids the pattern induction process by using clustering techniques on unlabelled pages. In this approach, a variant Hierarchical Agglomerative Clustering (HAC) algorithm called K-neighbours-HAC is developed which uses the similarities of the data format (HTML tags) and the data content (text string values) to group similar text tokens into clusters. We also develop a new method to label text tokens to capture the hierarchical structure of HTML pages and an algorithm for mapping labelled text tokens to XML. The new approach is tested and compared with several common existing wrapper induction systems on three different sets of web pages. The results suggest that the new approach is effective for data record detection and that it outperforms these common existing approaches examined on these web sites. Compared with the existing approaches, the new approach does not require training and successfully avoids the explicit pattern induction process, and accordingly the entire data detection process is simpler.