STEAMER: An interactive, inspectable, simulation-based training system
Artificial intelligence and instruction: Applications and methods
Representations of commonsense knowledge
Representations of commonsense knowledge
Automated generation of intent-based 3D Illustrations
Proceedings of the 18th annual conference on Computer graphics and interactive techniques
Graphics and natural language as components of automatic explanation
Intelligent user interfaces
Explanation and interaction: the computer generation of explanatory dialogues
Explanation and interaction: the computer generation of explanatory dialogues
Automated discourse generation using discourse structure relations
Artificial Intelligence - Special volume on natural language processing
Plan-based integration of natural language and graphics generation
Artificial Intelligence - Special volume on natural language processing
Automating the generation of coordinated multimedia explanations
Intelligent multimedia interfaces
Using argumentation to control lexical choice: a functional unification implementation
Using argumentation to control lexical choice: a functional unification implementation
Participating in explanatory dialogues: interpreting and responding to questions in context
Participating in explanatory dialogues: interpreting and responding to questions in context
Basic meanings of spatial relations: computation and evaluation in 3D space
AAAI'94 Proceedings of the twelfth national conference on Artificial intelligence (vol. 2)
Apex: An Experiment in the Automated Creation of Pictorial Explanations
IEEE Computer Graphics and Applications
Multimodal interactive maps: designing for human performance
Human-Computer Interaction
Generating explanatory captions for information graphics
IJCAI'95 Proceedings of the 14th international joint conference on Artificial intelligence - Volume 2
Realtime generation of customized 3D animated explanations for knowledge-based learning environments
AAAI'97/IAAI'97 Proceedings of the fourteenth national conference on artificial intelligence and ninth conference on Innovative applications of artificial intelligence
Coping with temporal constraints in multimedia presentation planning
AAAI'96 Proceedings of the thirteenth national conference on Artificial intelligence - Volume 1
Declarative camera control for automatic cinematography
AAAI'96 Proceedings of the thirteenth national conference on Artificial intelligence - Volume 1
Student-sensitive multimodal explanation generation for 3D learning environments
AAAI '99/IAAI '99 Proceedings of the sixteenth national conference on Artificial intelligence and the eleventh Innovative applications of artificial intelligence conference innovative applications of artificial intelligence
A hybrid indoor navigation system
Proceedings of the 6th international conference on Intelligent user interfaces
Automated authoring of coherent multimedia discourse in conversation systems
MULTIMEDIA '01 Proceedings of the ninth ACM international conference on Multimedia
IMPROVISE: Automated Generation of Animated Graphics for Coordinated Multimedia Presentations
CMC '98 Revised Papers from the Second International Conference on Cooperative Multimodal Communication
Architecture and implementation of multimodal plug and play
Proceedings of the 5th international conference on Multimodal interfaces
Deixis and conjunction in multimodal systems
COLING '00 Proceedings of the 18th conference on Computational linguistics - Volume 1
Taking account of the user's view in 3D multimodal instruction dialogue
COLING '00 Proceedings of the 18th conference on Computational linguistics - Volume 1
Towards integrated microplanning of language and iconic gesture for multimodal output
Proceedings of the 6th international conference on Multimodal interfaces
Compiling a lexicon of cooking actions for animation generation
COLING-ACL '06 Proceedings of the COLING/ACL on Main conference poster sessions
Generating approximate geographic descriptions
ENLG '09 Proceedings of the 12th European Workshop on Natural Language Generation
Generating approximate geographic descriptions
Empirical methods in natural language generation
| Hi-index | 0.00 |
Dynamically providing students with clear explanations of complex spatial concepts is critical for a broad range of knowledge-based educational and training systems. This calls for a realtime solution that can dynamically create 3D animated explanations that artfully integrate well-chosen speech with rich visualizations. Unfortunately, planning the integrated creation of 3D animation and spatial linguistic utterances in realtime requires coordinating the visual presentation of 3D objects and generating appropriate spatial phrases that accurately reflect the relative position, orientation, and direction of the objects presented. We present a visuo-linguistic framework for generating multimedia spatial explanations combining 3D animation and speech that complement one another. Because 3D animation planners require spatial knowledge in a geometric form and natural language generators require spatial knowledge in a linguistic form, a realtime multimedia planner interposed between the visual and linguistic components can serve as a mediator. This framework has been implemented in CINESPEAK, a multimedia explanation generator consisting of a visuo-linguistic mediator, a 3D animation planner, and a realtime natural language generator with a speech synthesizer. CINESPEAK has been used in conjunction with a prototype 3D learning environment in the domain of physics to generate realtime multimedia explanations of three dimensional electromagnetic fields, forces, and electrical current.