Model based vision as feedback for virtual reality robotics environments

• Authors: E. Natonek;T. Zimmerman;L. Fluckiger
• Venue: VRAIS '95 Proceedings of the Virtual Reality Annual International Symposium (VRAIS'95)
• Year: 1995

Task definition methods for robotic systems are often difficult to use. The "on-line" programming methods are often time expensive or risky for the human operator or the robot itself. On the other hand, "off-line" techniques are tedious and complex. In addition operator training is costly and time consuming. In a Virtual Reality Robotics Environment (VRRE), users are not asked to write down complicated functions, but can operate complex robotic systems in an intuitive and cost-effective way. However a VRRE is only effective if all the environment changes and object movements are fed-back to the virtual manipulating system. The paper describes the use of a VRRE for a semi-autonomous robot system comprising an industrial 5-axis robot, its virtual equivalent and a model based vision system used as feed-back. The user is immersed in a 3-D space built out of models of the robot's environment. He directly interacts with the virtual "components", defining tasks and dynamically optimizing them. A model based vision system locates objects in the real workspace to update the VRRE through a bi-directional communication link. In order to enhance the capabilities of the VRRE, a reflex-type behavior based on vision has been implemented. By locally (independently of the VRRE) controlling the real robot, the operator is discharged of small environmental changes due to transmission delays. Thus once the tasks have been optimized on the VRRE, they are sent to the real robot and a semi autonomous process ensures their correct execution thanks to a camera directly mounted on the robot's end effector. On the other hand if the environmental changes are too important, the robot stops, re-actualizes the VRRE with the new environmental configuration, and waits for task redesign. Because the operator interacts with the robotic system at a task oriented high level, VRRE systems are easily portable to other robotics environments (mobile robotics and micro assembly).