A framework for understanding latent semantic indexing (LSI) performance

Authors:
April Kontostathis;William M. Pottenger
Affiliations:
Ursinus College, Collegeville, PA;Lehigh University, Bethlehem, PA
Venue:
Information Processing and Management: an International Journal - Special issue: Formal methods for information retrieval
Year:
2006

Citing 15
Cited 10

Dimensions of meaning

Proceedings of the 1992 ACM/IEEE conference on Supercomputing
Using linear algebra for intelligent information retrieval

SIAM Review
An explanation of the effectiveness of latent semantic indexing by means of a Bayesian regression model

Information Processing and Management: an International Journal
Corpus-based stemming using cooccurrence of word variants

ACM Transactions on Information Systems (TOIS)
A similarity-based probability model for latent semantic indexing

Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval
Matrices, Vector Spaces, and Information Retrieval

SIAM Review
Level search schemes for information filtering and retrieval

Information Processing and Management: an International Journal
Using LSI for text classification in the presence of background text

Proceedings of the tenth international conference on Information and knowledge management
Information Retrieval

Information Retrieval
How Latent is Latent Semantic Analysis?

IJCAI '99 Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence
Connectionist interaction information retrieval

Information Processing and Management: an International Journal - Modelling vagueness and subjectivity in information access
Detecting emerging concepts in textual data mining

Computational information retrieval
SVDPACKC (Version 1.0) User''s Guide

SVDPACKC (Version 1.0) User''s Guide
Automatic word sense discrimination

Computational Linguistics - Special issue on word sense disambiguation
Choosing the word most typical in context using a lexical co-occurrence network

ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics

Combining text and link analysis for focused crawling-An application for vertical search engines

Information Systems
A similarity-based method for retrieving documents from the SCI/SSCI database

Journal of Information Science
Using position, fonts and cited references to retrieve scientific documents

Journal of Information Science
Query expansion and dimensionality reduction: Notions of optimality in Rocchio relevance feedback and latent semantic indexing

Information Processing and Management: an International Journal
An empirical study of required dimensionality for large-scale latent semantic indexing applications

Proceedings of the 17th ACM conference on Information and knowledge management
An analysis of latent semantic term self-correlation

ACM Transactions on Information Systems (TOIS)
A spectral-based clustering algorithm for categorical data using data summaries

Proceedings of the 2nd Workshop on Data Mining using Matrices and Tensors
Supervised latent semantic indexing using adaptive sprinkling

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Kernel latent semantic analysis using an information retrieval based kernel

Proceedings of the 18th ACM conference on Information and knowledge management
A higher order collective classifier for detecting andclassifying network events

ISI'09 Proceedings of the 2009 IEEE international conference on Intelligence and security informatics

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper we present a theoretical model for understanding the performance of Latent Semantic Indexing (LSI) search and retrieval application. Many models for understanding LSI have been proposed. Ours is the first to study the values produced by LSI in the term by dimension vectors. The framework presented here is based on term co-occurrence data. We show a strong correlation between second-order term co-occurrence and the values produced by the Singular Value Decomposition (SVD) algorithm that forms the foundation for LSI. We also present a mathematical proof that the SVD algorithm encapsulates term co-occurrence information.