The Strength of Weak Learnability
Machine Learning
Specifying gestures by example
Proceedings of the 18th annual conference on Computer graphics and interactive techniques
Boosting a weak learning algorithm by majority
Information and Computation
Recognition accuracy and user acceptance of pen interfaces
CHI '95 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Speech Communication - Special issue on speech under stress
A decision-theoretic generalization of on-line learning and an application to boosting
Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
Affective computing
Interpreting symptoms of cognitive load in speech input
UM '99 Proceedings of the seventh international conference on User modeling
Computers as Cognitive Tools: No More Walls
Computers as Cognitive Tools: No More Walls
Recognizing Time Pressure and Cognitive Load on the Basis of Speech: An Experimental Study
UM '01 Proceedings of the 8th International Conference on User Modeling 2001
Proceedings of HCI International (the 8th International Conference on Human-Computer Interaction) on Human-Computer Interaction: Ergonomics and User Interfaces-Volume I - Volume I
An Adaptive User Interface Based On Personalized Learning
IEEE Intelligent Systems
Modeling drivers' speech under stress
Speech Communication - Special issue on speech and emotion
HICSS '03 Proceedings of the 36th Annual Hawaii International Conference on System Sciences (HICSS'03) - Track 5 - Volume 5
Task-evoked pupillary response to mental workload in human-computer interaction
CHI '04 Extended Abstracts on Human Factors in Computing Systems
Exploiting prosodic structuring of coverbal gesticulation
Proceedings of the 6th international conference on Multimodal interfaces
When do we interact multimodally?: cognitive load and multimodal communication patterns
Proceedings of the 6th international conference on Multimodal interfaces
Assessing Cognitive Load with Physiological Sensors
HICSS '05 Proceedings of the Proceedings of the 38th Annual Hawaii International Conference on System Sciences - Volume 09
Human-centered design meets cognitive load theory: designing interfaces that help people think
MULTIMEDIA '06 Proceedings of the 14th annual ACM international conference on Multimedia
Examining the redundancy of multimodal input
OZCHI '06 Proceedings of the 18th Australia conference on Computer-Human Interaction: Design: Activities, Artefacts and Environments
Galvanic skin response (GSR) as an index of cognitive load
CHI '07 Extended Abstracts on Human Factors in Computing Systems
Using pen input features as indices of cognitive load
Proceedings of the 9th international conference on Multimodal interfaces
Automatic cognitive load detection from speech features
OZCHI '07 Proceedings of the 19th Australasian conference on Computer-Human Interaction: Entertaining User Interfaces
I'm sad you're sad: emotional contagion in CMC
Proceedings of the 2008 ACM conference on Computer supported cooperative work
Multimodal interactive maps: designing for human performance
Human-Computer Interaction
Cognitive Load Measurement from User's Linguistic Speech Features for Adaptive Interaction Design
INTERACT '09 Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part I
Using language complexity to measure cognitive load for adaptive interaction design
Proceedings of the 15th international conference on Intelligent user interfaces
Towards automatic cognitive load measurement from speech analysis
HCI'07 Proceedings of the 12th international conference on Human-computer interaction: interaction design and usability
The significance of empty speech pauses: cognitive and algorithmic issues
BVAI'07 Proceedings of the 2nd international conference on Advances in brain, vision and artificial intelligence
A Study of Voice Source and Vocal Tract Filter Based Features in Cognitive Load Classification
ICPR '10 Proceedings of the 2010 20th International Conference on Pattern Recognition
Cognitive load evaluation of handwriting using stroke-level features
Proceedings of the 16th international conference on Intelligent user interfaces
Investigation of spectral centroid features for cognitive load classification
Speech Communication
Formant frequencies under cognitive load: effects and classification
EURASIP Journal on Advances in Signal Processing - Special issue on emotion and mental state recognition from speech
Upset now?: emotion contagion in distributed groups
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Freeform pen-input as evidence of cognitive load and expertise
ICMI '11 Proceedings of the 13th international conference on multimodal interfaces
Teaching athletes cognitive skills: detecting cognitive load in speech input
BCS '10 Proceedings of the 24th BCS Interaction Specialist Group Conference
Assessing recovery from cognitive load through pen input
CHI '13 Extended Abstracts on Human Factors in Computing Systems
CoWME: a general framework to evaluate cognitive workload during multimodal interaction
Proceedings of the 15th ACM on International conference on multimodal interaction
Analysing mouse activity for cognitive load detection
Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration
Review Article: Multimodal interaction: A review
Pattern Recognition Letters
Hi-index | 0.00 |
High cognitive load arises from complex time and safety-critical tasks, for example, mapping out flight paths, monitoring traffic, or even managing nuclear reactors, causing stress, errors, and lowered performance. Over the last five years, our research has focused on using the multimodal interaction paradigm to detect fluctuations in cognitive load in user behavior during system interaction. Cognitive load variations have been found to impact interactive behavior: by monitoring variations in specific modal input features executed in tasks of varying complexity, we gain an understanding of the communicative changes that occur when cognitive load is high. So far, we have identified specific changes in: speech, namely acoustic, prosodic, and linguistic changes; interactive gesture; and digital pen input, both interactive and freeform. As ground-truth measurements, galvanic skin response, subjective, and performance ratings have been used to verify task complexity. The data suggest that it is feasible to use features extracted from behavioral changes in multiple modal inputs as indices of cognitive load. The speech-based indicators of load, based on data collected from user studies in a variety of domains, have shown considerable promise. Scenarios include single-user and team-based tasks; think-aloud and interactive speech; and single-word, reading, and conversational speech, among others. Pen-based cognitive load indices have also been tested with some success, specifically with pen-gesture, handwriting, and freeform pen input, including diagraming. After examining some of the properties of these measurements, we present a multimodal fusion model, which is illustrated with quantitative examples from a case study. The feasibility of employing user input and behavior patterns as indices of cognitive load is supported by experimental evidence. Moreover, symptomatic cues of cognitive load derived from user behavior such as acoustic speech signals, transcribed text, digital pen trajectories of handwriting, and shapes pen, can be supported by well-established theoretical frameworks, including O'Donnell and Eggemeier's workload measurement [1986] Sweller's Cognitive Load Theory [Chandler and Sweller 1991], and Baddeley's model of modal working memory [1992] as well as McKinstry et al.'s [2008] and Rosenbaum's [2005] action dynamics work. The benefit of using this approach to determine the user's cognitive load in real time is that the data can be collected implicitly that is, during day-to-day use of intelligent interactive systems, thus overcomes problems of intrusiveness and increases applicability in real-world environments, while adapting information selection and presentation in a dynamic computer interface with reference to load.