Evaluating maps produced by urban search and rescue robots: lessons learned from RoboCup

Authors:
Benjamin Balaguer;Stephen Balakirsky;Stefano Carpin;Arnoud Visser
Affiliations:
School of Engineering, University of California, Merced, Merced, USA 95343;Intelligent Systems Division, National Institute of Standards and Technology, Gaithersburg, USA 20899-8230;School of Engineering, University of California, Merced, Merced, USA 95343;Intelligent System Laboratory Amsterdam, Universiteit van Amsterdam, Amsterdam, The Netherlands 1098
Venue:
Autonomous Robots
Year:
2009

Citing 8
Cited 7

Sensor fusion in certainty grids for mobile robots

AI Magazine
Feature Extraction: Foundations and Applications (Studies in Fuzziness and Soft Computing)

Feature Extraction: Foundations and Applications (Studies in Fuzziness and Soft Computing)
Towards heterogeneous robot teams for disaster mitigation: Results and performance metrics from RoboCup rescue: Field Reports

Journal of Field Robotics - Special Issue on Teamwork in Field Robotics
Planetary Exploration in USARsim: A Case Study Including Real World Data from Mars

RoboCup 2008: Robot Soccer World Cup XII
Exploring unknown environments with mobile robots using coverage maps

IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Evaluating techniques for resolving redundant information and specularity in occupancy grids

AI'05 Proceedings of the 18th Australian Joint conference on Advances in Artificial Intelligence
High fidelity tools for rescue robotics: results and perspectives

RoboCup 2005
Improved Techniques for Grid Mapping With Rao-Blackwellized Particle Filters

IEEE Transactions on Robotics

Evaluating the RoboCup 2009 Virtual Robot Rescue Competition

PerMIS '09 Proceedings of the 9th Workshop on Performance Metrics for Intelligent Systems
Evaluation of RoboCup maps

PerMIS '09 Proceedings of the 9th Workshop on Performance Metrics for Intelligent Systems
Exploration strategies based on multi-criteria decision making for search and rescue autonomous robots

The 10th International Conference on Autonomous Agents and Multiagent Systems - Volume 1
Exploration strategies based on multi-criteria decision making for searching environments in rescue operations

Autonomous Robots
Evaluation criteria for appearance based maps

Proceedings of the 10th Performance Metrics for Intelligent Systems Workshop
Evaluation of maps using fixed shapes: the fiducial map metric

Proceedings of the 10th Performance Metrics for Intelligent Systems Workshop
Safety and Precision of Spatial Context Models for Autonomous Systems

Electronic Notes in Theoretical Computer Science (ENTCS)

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents the map evaluation methodology developed for the Virtual Robots Rescue competition held as part of RoboCup. The procedure aims to evaluate the quality of maps produced by multi-robot systems with respect to a number of factors, including usability, exploration, annotation and other aspects relevant to robots and first responders. In addition to the design choices, we illustrate practical examples of maps and scores coming from the latest RoboCup contest, outlining strengths and weaknesses of our modus operandi. We also show how a benchmarking methodology developed for a simulation testbed effortlessly and faithfully transfers to maps built by a real robot. A number of conclusions may be derived from the experience reported in this paper and a thorough discussion is offered.