"Turn Off the Television!": Real-World Robotic Exploration Experiments with a Virtual 3-D Display
HICSS '05 Proceedings of the Proceedings of the 38th Annual Hawaii International Conference on System Sciences - Volume 09
AAAI'04 Proceedings of the 19th national conference on Artifical intelligence
Shared understanding for collaborative control
IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans
| Hi-index | 0.00 |
This paper discusses approaches developed at the Idaho National Laboratory (INL) for quantifying and analyzing the performance of human-robot teams across different domains. These methods reflect experience and insights gained from previous INL experiments that have focused on landmine detection and marking; mapping and localization for robot positioning, mobile manipulation for explosive ordinance disposal (EOD), radiation characterization, and urban search and rescue operations. An overarching goal of this work has been to enhance our understanding of how the robot, the control and display interface, the task context, and the human contribute to or hinder mission success. Our approach to performance measurement was developed in concert with the iterative design cycle of our intelligent robotic control system, the robot intelligence kernel (RIK). In extending and refining the RIK for various applications, three factors key to holistic human-robot performance assessment were identified: comprehensive planning; the inclusion of end users in the design and performance evaluation phases of the study; and combining automated data collection with subjective measures. The paper discusses lessons learned in developing and applying performance metrics and provides a brief overview of measures that we are currently using to support the assessment of autonomy. In particular, the paper emphasizes the application of these metrics to behaviors for complex and potentially dangerous missions.