Common metrics for human-robot interaction

Authors:
Aaron Steinfeld;Terrence Fong;David Kaber;Michael Lewis;Jean Scholtz;Alan Schultz;Michael Goodrich
Affiliations:
Carnegie Mellon University, Pittsburgh, PA;NASA Ames Research Center, Moffett Field, CA;North Carolina State University, Raleigh, NC;University of Pittsburgh, Pittsburgh, PA;NIST, Gaithersburg, MD;US Naval Research Lab, Washington, DC;Brigham Young University, Provo, UT
Venue:
Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction
Year:
2006

Citing 21
Cited 49

Telerobotics, automation, and human supervisory control

Telerobotics, automation, and human supervisory control
Lag as a determinant of human performance in interactive systems

INTERCHI '93 Proceedings of the INTERCHI '93 conference on Human factors in computing systems
Trust, self-confidence, and operators' adaptation to automation

International Journal of Human-Computer Studies
Cooperation with a robotic assistant

CHI '02 Extended Abstracts on Human Factors in Computing Systems
Online robots and the remote museum experience

Beyond webcams
Engineering Psychology and Cognitive Ergonomics: Volume Six

Engineering Psychology and Cognitive Ergonomics: Volume Six
Interactive System Design

Interactive System Design
Socially Intelligent Agents: Creating Relationships with Computers and Robots

Socially Intelligent Agents: Creating Relationships with Computers and Robots
Vehicle Teleoperation Interfaces

Autonomous Robots
A New Method to Evaluate Human-Robot System Performance

Autonomous Robots
A Reference Test Course for Urban Search and Rescue Robots

Proceedings of the Fourteenth International Florida Artificial Intelligence Research Society Conference
Theory and Evaluation of Human Robot Interactions

HICSS '03 Proceedings of the 36th Annual Hawaii International Conference on System Sciences (HICSS'03) - Track 5 - Volume 5
Evaluation of a Human-Robot Interface: Development of a Situational Awareness Methodology

HICSS '04 Proceedings of the Proceedings of the 37th Annual Hawaii International Conference on System Sciences (HICSS'04) - Track 5 - Volume 5
Robotic camera control for remote exploration

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Control gain adaptation in virtual reality mediated human–telerobot interaction: Research Articles

Human Factors in Ergonomics & Manufacturing
Adapting GOMS to model human-robot interaction

Proceedings of the ACM/IEEE international conference on Human-robot interaction
The effects of network delays on group work in real-time groupware

ECSCW'01 Proceedings of the seventh conference on European Conference on Computer Supported Cooperative Work
Beyond usability evaluation: analysis of human-robot interaction at a major robotics competition

Human-Computer Interaction
Autonomy levels for unmanned systems (ALFUS) framework: safety and application issues

PerMIS '07 Proceedings of the 2007 Workshop on Performance Metrics for Intelligent Systems
Human-robot interactions during the robot-assisted urban search and rescue response at the World Trade Center

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
A model for types and levels of human interaction with automation

IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans

Developing performance metrics for the supervisory control of multiple robots

Proceedings of the ACM/IEEE international conference on Human-robot interaction
A field experiment of autonomous mobility: operator workload for one and two robots

Proceedings of the ACM/IEEE international conference on Human-robot interaction
Assessing the scalability of a multiple robot interface

Proceedings of the ACM/IEEE international conference on Human-robot interaction
Young researchers' views on the current and future state of HRI

Proceedings of the ACM/IEEE international conference on Human-robot interaction
Human-robot interaction: a survey

Foundations and Trends in Human-Computer Interaction
Assessing cooperation in human control of heterogeneous robots

Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction
Beyond dirty, dangerous and dull: what everyday people think robots should do

Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction
Housewives or technophiles?: understanding domestic robot owners

Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction
Simultaneous teleoperation of multiple social robots

Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction
The Experimental Robotics Framework

SIMPAR '08 Proceedings of the 1st International Conference on Simulation, Modeling, and Programming for Autonomous Robots
Systemic interaction analysis (SInA) in HRI

Proceedings of the 4th ACM/IEEE international conference on Human robot interaction
The oz of wizard: simulating the human for interaction research

Proceedings of the 4th ACM/IEEE international conference on Human robot interaction
Field trial for simultaneous teleoperation of mobile social robots

Proceedings of the 4th ACM/IEEE international conference on Human robot interaction
A Scaled Bilateral Control System for Experimental One-dimensional Teleoperated Nanomanipulation

International Journal of Robotics Research
I'm sorry, Dave: i'm afraid i won't do that: social aspects of human-agent conflict

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
A survey of robot learning from demonstration

Robotics and Autonomous Systems
A spotlight on security and privacy risks with future household robots: attacks and lessons

Proceedings of the 11th international conference on Ubiquitous computing
Human system performance metrics for evaluation of mixed-initiative heterogeneous autonomous systems

PerMIS '07 Proceedings of the 2007 Workshop on Performance Metrics for Intelligent Systems
Mobility reconfiguration for terrain exploration using passive perception

ICRA'09 Proceedings of the 2009 IEEE international conference on Robotics and Automation
Human performance moderator functions for human-robot peer-based teams

Proceedings of the 5th ACM/IEEE international conference on Human-robot interaction
Lead me by the hand: evaluation of a direct physical interface for nursing assistant robots

Proceedings of the 5th ACM/IEEE international conference on Human-robot interaction
Three way relationship of human-robot interaction

HCI'07 Proceedings of the 12th international conference on Human-computer interaction: intelligent multimodal interaction environments
Evaluation criteria for human-automation performance metrics

PerMIS '08 Proceedings of the 8th Workshop on Performance Metrics for Intelligent Systems
Assessing measures of coordination demand based on interaction durations

PerMIS '08 Proceedings of the 8th Workshop on Performance Metrics for Intelligent Systems
The gestural joystick and the efficacy of the path tortuosity metric for human/robot interaction

PerMIS '08 Proceedings of the 8th Workshop on Performance Metrics for Intelligent Systems
Robotic systems technical and operational metrics correlation

PerMIS '08 Proceedings of the 8th Workshop on Performance Metrics for Intelligent Systems
Measurement techniques for multiagent systems

PerMIS '08 Proceedings of the 8th Workshop on Performance Metrics for Intelligent Systems
A survey of Tactile Human-Robot Interactions

Robotics and Autonomous Systems
On the architecture of a human-centered CAD agent system

Computer-Aided Design
Usability of force-based controllers in physical human-robot interaction

Proceedings of the 6th international conference on Human-robot interaction
Nonverbal robot-group interaction using an imitated gaze cue

Proceedings of the 6th international conference on Human-robot interaction
Integrated simulation environment for unmanned autonomous systems: towards a conceptual framework

Modelling and Simulation in Engineering
Validating extended neglect tolerance model for human robot interactions in humanoid soccer robots

Robotica
Human sensorimotor learning for humanoid robot skill synthesis

Adaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems
Evaluating human-robot interaction during a manipulation experiment conducted in immersive virtual reality

Proceedings of the 2011 international conference on Virtual and mixed reality: new trends - Volume Part I
End-user programming of a social robot by dialog

Robotics and Autonomous Systems
RoboCup@Home: results in benchmarking domestic service robots

RoboCup 2009
Modeling prehension for physical collaboration in virtual environments

Presence: Teleoperators and Virtual Environments
Evaluating autonomous ground-robots

Journal of Field Robotics
Metric selection for evaluating human supervisory control of unmanned vehicles

Proceedings of the 10th Performance Metrics for Intelligent Systems Workshop
Observations on single operator performance controlling two UGVs in a field assessment

Proceedings of the 10th Performance Metrics for Intelligent Systems Workshop
HRI evaluation of a healthcare service robot

ICSR'12 Proceedings of the 4th international conference on Social Robotics
A robot learning from demonstration framework to perform force-based manipulation tasks

Intelligent Service Robotics
Survey of metrics for human-robot interaction

Proceedings of the 8th ACM/IEEE international conference on Human-robot interaction
Performance effects of multi-sensory displays in virtual teleoperation environments

Proceedings of the 1st symposium on Spatial user interaction
A precursory look at potential interaction objectives affecting flexible robotic cell safety

HCI International'13 Proceedings of the 15th international conference on Human Interface and the Management of Information: information and interaction for health, safety, mobility and complex environments - Volume Part II
Task fidelity for robot-mediated interaction in 3D worlds

Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction
Model driven software development for human-machine interaction systems

Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction
Effects of a social robot's autonomy and group orientation on human decision-making

Advances in Human-Computer Interaction

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper describes an effort to identify common metrics for task-oriented human-robot interaction (HRI). We begin by discussing the need for a toolkit of HRI metrics. We then describe the framework of our work and identify important biasing factors that must be taken into consideration. Finally, we present suggested common metrics for standardization and a case study. Preparation of a larger, more detailed toolkit is in progress.