WordsEye: an automatic text-to-scene conversion system
Proceedings of the 28th annual conference on Computer graphics and interactive techniques
Automatic labeling of semantic roles
Computational Linguistics
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Natural language input for scene generation
EACL '83 Proceedings of the first conference on European chapter of the Association for Computational Linguistics
COLING '98 Proceedings of the 17th international conference on Computational linguistics - Volume 1
Design considerations for Picture Production in a Natural Language graphics system
ACM SIGGRAPH Computer Graphics
Reconstructing spatial image from natural language texts
COLING '92 Proceedings of the 14th conference on Computational linguistics - Volume 4
Motivations and methods for text simplification
COLING '96 Proceedings of the 16th conference on Computational linguistics - Volume 2
BLEU: a method for automatic evaluation of machine translation
ACL '02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
Accurate unlexicalized parsing
ACL '03 Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - Volume 1
The Proposition Bank: An Annotated Corpus of Semantic Roles
Computational Linguistics
The Story Picturing Engine---a system for automatic text illustration
ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP)
Put: Language-Based Interactive Manipulation of Objects
IEEE Computer Graphics and Applications
Broad-coverage sense disambiguation and information extraction with a supersense sequence tagger
EMNLP '06 Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing
A text-to-picture synthesis system for augmenting communication
AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 2
Meteor: an automatic metric for MT evaluation with high levels of correlation with human judgments
StatMT '07 Proceedings of the Second Workshop on Statistical Machine Translation
Automatic text-to-scene conversion in the traffic accident domain
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Multimodal summarization of complex sentences
Proceedings of the 16th international conference on Intelligent user interfaces
Kinect-based visual communication system
Proceedings of the 4th International Conference on Internet Multimedia Computing and Service
Hi-index | 0.00 |
Pictorial communication systems convert natural language text into pictures to assist people with limited literacy. We define a novel and challenging problem: picture layout optimization. Given an input sentence, we seek the optimal way to lay out word icons such that the resulting picture best conveys the meaning of the input sentence. To this end, we propose a family of intuitive "ABC" layouts, which organize icons in three groups. We formalize layout optimization as a sequence labeling problem, employing conditional random fields as our machine learning method. Enabled by novel applications of semantic role labeling and syntactic parsing, our trained model makes layout predictions that agree well with human annotators. In addition, we conduct a user study to compare our ABC layout versus the standard linear layout. The study shows that our semantically enhanced layout is preferred by non-native speakers, suggesting it has the potential to be useful for people with other forms of limited literacy, too.