Developing multimodal web interfaces by encapsulating their content and functionality within a multimodal shell

  • Authors:
  • Izidor Mlakar;Matej Rojc

  • Affiliations:
  • Roboti c.s. d.o.o, Slovenia;Faculty of Electrical Engineering and Computer Science, University of Maribor, Slovenia

  • Venue:
  • COST'10 Proceedings of the 2010 international conference on Analysis of Verbal and Nonverbal Communication and Enactment
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Web applications are a widely-spread and a widely-used concept for presenting information. Their underlying architecture and standards, in many cases, limit their presentation/control capabilities of showing pre-recorded audio/video sequences. Highly-dynamic text content, for instance, can only be displayed in its native from (as part of HTML content). This paper provides concepts and answers that enable the transformation of dynamic web-based content into multimodal sequences generated by different multimodal services. Based on the encapsulation of the content into a multimodal shell, any text-based data can dynamically and at interactive speeds be transformed into multimodal visually-synthesized speech. Techniques for the integration of multimodal input (e.g. visioning and speech recognition) are also included. The concept of multimodality relies on mashup approaches rather than traditional integration. It can, therefore, extended any type of web-based solution transparently with no major changes to either the multimodal services or the enhanced web-application.