Linear scale semantic mining algorithms in microsoft SQL server's semantic platform

Authors:
Kunal Mukerjee;Todd Porter;Sorin Gherman
Affiliations:
Microsoft, Redmond, WA, USA;Microsoft, Redmond, WA, USA;Microsoft, Redmond, WA, USA
Venue:
Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining
Year:
2011

Citing 10
Cited 0

Highlights: language- and domain-independent automatic indexing terms for abstracting

Journal of the American Society for Information Science
Web document clustering: a feasibility demonstration

Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval
KEA: practical automatic keyphrase extraction

Proceedings of the fourth ACM conference on Digital libraries
A vector space model for automatic indexing

Communications of the ACM
Modern Information Retrieval

Modern Information Retrieval
Database Research: Achievements and Opportunities into the 21st Century

Database Research: Achievements and Opportunities into the 21st Century
Latent dirichlet allocation

The Journal of Machine Learning Research
Document Similarity Using a Phrase Indexing Graph Model

Knowledge and Information Systems
Introduction to Data Mining, (First Edition)

Introduction to Data Mining, (First Edition)
Computing semantic relatedness using Wikipedia-based explicit semantic analysis

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper describes three linear scale, incremental, and fully automatic semantic mining algorithms that are at the foundation of the new Semantic Platform being released in the next version of SQL Server. The target workload is large (10 -- 100 million) Enterprise document corpuses. At these scales, anything short of linear scale and incremental is costly to deploy. These three algorithms give rise to three weighted physical indexes: Tag Index (top keywords in each document); Document Similarity Index (top closely related documents given any document); and Semantic Phrase Similarity Index (top semantically related phrases, given any phrase), which are then query-able through the SQL interface. The need for specifically creating these three indexes was motivated by observing typical stages of document research, and gap analysis, given current tools and technology at the Enterprise. We describe the mining algorithms and architecture, and also outline some compelling user experiences that are enabled by the indexes.