Approximate Kalman filters for embedding author-word co-occurrence data over time

Authors:
Purnamrita Sarkar;Sajid M. Siddiqi;Geoffrey J. Gordon
Affiliations:
Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA;Robotics Institute, Carnegie Mellon University, Pittsburgh, PA;Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA
Venue:
ICML'06 Proceedings of the 2006 conference on Statistical network analysis
Year:
2006

Citing 2
Cited 0

Computational Methods for Intelligent Information Access

Supercomputing '95 Proceedings of the 1995 ACM/IEEE conference on Supercomputing
An Introduction to Variational Methods for Graphical Models

Machine Learning

Quantified Score

Hi-index	0.01

Visualization

Abstract

We address the problem of embedding entities into Euclidean space over time based on co-occurrence data. We extend the CODE model of [1] to a dynamic setting. This leads to a non-standard factored state space model with real-valued hidden parent nodes and discrete observation nodes. We investigate the use of variational approximations applied to the observation model that allow us to formulate the entire dynamic model as a Kalman filter. Applying this model to temporal co-occurrence data yields posterior distributions of entity coordinates in Euclidean space that are updated over time. Initial results on per-year co-occurrences of authors and words in the NIPS corpus and on synthetic data, including videos of dynamic embeddings, seem to indicate that the model results in embeddings of co-occurrence data that are meaningful both temporally and contextually.