A language-based approach to indexing heterogeneous multimedia lifelog

Authors:
Peng-Wen Cheng;Snehal Chennuru;Senaka Buthpitiya;Ying Zhang
Affiliations:
Carnegie Mellon Silicon Valley, NASA Ames Research Park, Moffett Feild, CA;Carnegie Mellon Silicon Valley, NASA Ames Research Park, Moffett Feild, CA;Carnegie Mellon Silicon Valley, NASA Ames Research Park, Moffett Feild, CA;Carnegie Mellon Silicon Valley, NASA Ames Research Park, Moffett Feild, CA
Venue:
International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction
Year:
2010

Citing 7
Cited 0

Suffix arrays: a new method for on-line string searches

SIAM Journal on Computing
MyLifeBits: fulfilling the Memex vision

Proceedings of the tenth ACM international conference on Multimedia
Efficient retrieval of life log based on context and content

Proceedings of the the 1st ACM workshop on Continuous archival and retrieval of personal experiences
Trajectory pattern mining

Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
Modeling and Analyzing Individual's Daily Activities using Lifelog

ICESS '08 Proceedings of the 2008 International Conference on Embedded Software and Systems
Personalized life log media system in ubiquitous environment

ICUCT'06 Proceedings of the 1st international conference on Ubiquitous convergence technology
Modeling human behavior from simple sensors in the home

PERVASIVE'06 Proceedings of the 4th international conference on Pervasive Computing

Quantified Score

Hi-index	0.01

Visualization

Abstract

Lifelog systems, inspired by Vannevar Bush's concept of "MEMory EXtenders" (MEMEX), are capable of storing a person's lifetime experience as a multimedia database. Despite such systems' huge potential for improving people's everyday life, there are major challenges that need to be addressed to make such systems practical. One of them is how to index the inherently large and heterogeneous lifelog data so that a person can efficiently retrieve the log segments that are of interest. In this paper, we present a novel approach to indexing lifelogs using activity language. By quantizing the heterogeneous high dimensional sensory data into text representation, we are able to apply statistical natural language processing techniques to index, recognize, segment, cluster, retrieve, and infer high-level semantic meanings of the collected lifelogs. Based on this indexing approach, our lifelog system supports easy retrieval of log segments representing past similar activities and generation of salient summaries serving as overviews of segments.