Sensing cognitive multitasking for a brain-based adaptive user interface

Authors:
Erin Treacy Solovey;Francine Lalooses;Krysta Chauncey;Douglas Weaver;Margarita Parasi;Matthias Scheutz;Angelo Sassaroli;Sergio Fantini;Paul Schermerhorn;Audrey Girouard;Robert J.K. Jacob
Affiliations:
Tufts University, Medford, Massachusetts, USA;Tufts University, Medford, Massachusetts, USA;Tufts University, Medford, Massachusetts, USA;Tufts University, Medford, Massachusetts, USA;Tufts University, Medford, Massachusetts, USA;Tufts University, Medford, Massachusetts, USA;Tufts University, Medford, Massachusetts, USA;Tufts University, Medford, Massachusetts, USA;Indiana University, Bloomington, Indiana, USA;Queen's University, Kingston, Ontario, Canada;Tufts University, Medford, Massachusetts, USA
Venue:
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Year:
2011

Citing 25
Cited 5

Toward Machine Emotional Intelligence: Analysis of Affective Physiological State

IEEE Transactions on Pattern Analysis and Machine Intelligence - Graph Algorithms and Computer Vision
Human-Computer Interaction INTERACT '99: IFIP TC. 13 - 1999 Edinburgh

Human-Computer Interaction INTERACT '99: IFIP TC. 13 - 1999 Edinburgh
Predicting human interruptibility with sensors: a Wizard of Oz feasibility study

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Visual Contextual Awareness in Wearable Computing

ISWC '98 Proceedings of the 2nd IEEE International Symposium on Wearable Computers
Examining the robustness of sensor-based statistical models of human interruptibility

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Task-evoked pupillary response to mental workload in human-computer interaction

CHI '04 Extended Abstracts on Human Factors in Computing Systems
Physiological indicators for the evaluation of co-located collaborative play

CSCW '04 Proceedings of the 2004 ACM conference on Computer supported cooperative work
Towards an index of opportunity: understanding changes in mental workload during task execution

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Investigating the effectiveness of mental workload as a predictor of opportune moments for interruption

CHI '05 Extended Abstracts on Human Factors in Computing Systems
Using a low-cost electroencephalograph for task classification in HCI research

UIST '06 Proceedings of the 19th annual ACM symposium on User interface software and technology
First steps toward natural human-like HRI

Autonomous Robots
Feasibility and pragmatics of classifying working memory load with an electroencephalograph

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Comparison of four primary methods for coordinating the interruption of people in human-computer interaction

Human-Computer Interaction
Fundamentals of physiological computing

Interacting with Computers
Brain measurement for usability testing and adaptive interfaces: an example of uncovering syntactic workload with functional near infrared spectroscopy

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Using fNIRS brain sensing in realistic HCI settings: experiments and guidelines

Proceedings of the 22nd annual ACM symposium on User interface software and technology
Dynamic robot autonomy: investigating the effects of robot decision-making in a human-robot team task

Proceedings of the 2009 international conference on Multimodal interfaces
The WEKA data mining software: an update

ACM SIGKDD Explorations Newsletter
Multitasking and monotasking: the effects of mental workload on deferred task interruptions

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
A novel brain-computer interface using a multi-touch surface

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
The influence of implicit and explicit biofeedback in first-person shooter games

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Knowing where and when to look in a time-critical multimodal dual task

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Towards a physiological model of user interruptability

INTERACT'07 Proceedings of the 11th IFIP TC 13 international conference on Human-computer interaction - Volume Part II
A model for types and levels of human interaction with automation

IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans
Designing a passive brain computer interface using real time classification of functional near-infrared spectroscopy

International Journal of Autonomous and Adaptive Communications Systems

Brainput: enhancing interactive systems with streaming fnirs brain input

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Construction of the biocybernetic loop: a case study

Proceedings of the 14th ACM international conference on Multimodal interaction
Using fNIRS brain sensing to evaluate information visualization interfaces

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Exploring factors influencing multitasking interaction with multiple smart devices

Computers in Human Behavior
FRIENDs: Brain-monitoring agents for adaptive socio-technical systems

Multiagent and Grid Systems

Quantified Score

Hi-index	0.01

Visualization

Abstract

Multitasking has become an integral part of work environments, even though people are not well-equipped cognitively to handle numerous concurrent tasks effectively. Systems that support such multitasking may produce better performance and less frustration. However, without understanding the user's internal processes, it is difficult to determine optimal strategies for adapting interfaces, since all multitasking activity is not identical. We describe two experiments leading toward a system that detects cognitive multitasking processes and uses this information as input to an adaptive interface. Using functional near-infrared spectroscopy sensors, we differentiate four cognitive multitasking processes. These states cannot readily be distinguished using behavioral measures such as response time, accuracy, keystrokes or screen contents. We then present our human-robot system as a proof-of-concept that uses real-time cognitive state information as input and adapts in response. This prototype system serves as a platform to study interfaces that enable better task switching, interruption management, and multitasking.