WordsEye: an automatic text-to-scene conversion system
Proceedings of the 28th annual conference on Computer graphics and interactive techniques
Multimedia Learning
Summarization beyond sentence extraction: a probabilistic approach to sentence compression
Artificial Intelligence
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
The Journal of Machine Learning Research
The Story Picturing Engine---a system for automatic text illustration
ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP)
Image retrieval: Ideas, influences, and trends of the new age
ACM Computing Surveys (CSUR)
Toward communicating simple sentences using pictorial representations
Machine Translation
Easy as ABC?: facilitating pictorial communication via semantically enhanced layout
CoNLL '08 Proceedings of the Twelfth Conference on Computational Natural Language Learning
A text-to-picture synthesis system for augmenting communication
AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 2
SemEval-2010 task 13: evaluating events, time expressions, and temporal relations (TempEval-2)
DEW '09 Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions
Deep semantic analysis of text
STEP '08 Proceedings of the 2008 Conference on Semantics in Text Processing
TRIPS and TRIOS system for TempEval-2: Extracting temporal information from text
SemEval '10 Proceedings of the 5th International Workshop on Semantic Evaluation
Automatic generation of social media snippets for mobile browsing
Proceedings of the 21st ACM international conference on Multimedia
| Hi-index | 0.00 |
In this paper, we introduce the idea of automatically illustrating complex sentences as multimodal summaries that combine pictures, structure and simplified compressed text. By including text and structure in addition to pictures, multimodal summaries provide additional clues of what happened, who did it, to whom and how, to people who may have difficulty reading or who are looking to skim quickly. We present ROC-MMS, a system for automatically creating multimodal summaries (MMS) of complex sentences by generating pictures, textual summaries and structure. We show that pictures alone are insufficient to help people understand most sentences, especially for readers who are unfamiliar with the domain. An evaluation of ROC-MMS in the Wikipedia domain illustrates both the promise and challenge of automatically creating multimodal summaries.