Prototyping novel collaborative multimodal systems: simulation, data collection and analysis tools for the next decade

  • Authors:
  • Alexander M. Arthur;Rebecca Lunsford;Matt Wesson;Sharon Oviatt

  • Affiliations:
  • Natural Interaction Systems LLC, Seattle, WA;Natural Interaction Systems LLC, Seattle, WA and Oregon Health & Science Uni., Beaverton, OR;Natural Interaction Systems LLC, Seattle, WA;Natural Interaction Systems LLC, Seattle, WA and Oregon Health & Science Uni., Beaverton, OR and University of Washington, Seattle, WA

  • Venue:
  • Proceedings of the 8th international conference on Multimodal interfaces
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

To support research and development of next-generation multimodal interfaces for complex collaborative tasks, a comprehensive new infrastructure has been created for collecting and analyzing time-synchronized audio, video, and pen-based data during multi-party meetings. This infrastructure needs to be unobtrusive and to collect rich data involving multiple information sources of high temporal fidelity to allow the collection and annotation of simulation-driven studies of natural human-human-computer interactions. Furthermore, it must be flexibly extensible to facilitate exploratory research. This paper describes both the infrastructure put in place to record, encode, playback and annotate the meeting-related media data, and also the simulation environment used to prototype novel system concepts.