Purposive behavior acquisition for a real robot by vision-based reinforcement learning
Machine Learning - Special issue on robot learning
Visual Homing: Surfing on the Epipoles
International Journal of Computer Vision
Reinforcement learning for landmark-based robot navigation
Proceedings of the first international joint conference on Autonomous agents and multiagent systems: part 2
Target Reaching by Using Visual Information and Q-learning Controllers
Autonomous Robots
Image-based robot navigation from an image memory
Robotics and Autonomous Systems
Speeded-Up Robust Features (SURF)
Computer Vision and Image Understanding
A reinforcement agent for object segmentation in ultrasound images
Expert Systems with Applications: An International Journal
Case-based reinforcement learning for dynamic inventory control in a multi-agent supply-chain system
Expert Systems with Applications: An International Journal
Image Based and Hybrid Visual Servo Control of an Unmanned Aerial Vehicle
Journal of Intelligent and Robotic Systems
Reinforcement learning approach to goal-regulation in a self-evolutionary manufacturing system
Expert Systems with Applications: An International Journal
Keeping features in the field of view in eye-in-hand visual servoing: a switching approach
IEEE Transactions on Robotics
Ensuring visibility in calibration-free path planning for image-based visual servoing
IEEE Transactions on Robotics
Image-Based Visual Servoing for Nonholonomic Mobile Robots Using Epipolar Geometry
IEEE Transactions on Robotics
Global Path-Planning for Constrained and Optimal Visual Servoing
IEEE Transactions on Robotics
| Hi-index | 12.05 |
It is known that most of the key problems in visual servo control of robots are related to the performance analysis of the system considering measurement and modeling errors. In this paper, the development and performance evaluation of a novel intelligent visual servo controller for a robot manipulator using neural network Reinforcement Learning is presented. By implementing machine learning techniques into the vision based control scheme, the robot is enabled to improve its performance online and to adapt to the changing conditions in the environment. Two different temporal difference algorithms (Q-learning and SARSA) coupled with neural networks are developed and tested through different visual control scenarios. A database of representative learning samples is employed so as to speed up the convergence of the neural network and real-time learning of robot behavior. Moreover, the visual servoing task is divided into two steps in order to ensure the visibility of the features: in the first step centering behavior of the robot is conducted using neural network Reinforcement Learning controller, while the second step involves switching control between the traditional Image Based Visual Servoing and the neural network Reinforcement Learning for enabling approaching behavior of the manipulator. The correction in robot motion is achieved with the definition of the areas of interest for the image features independently in both control steps. Various simulations are developed in order to present the robustness of the developed system regarding calibration error, modeling error, and image noise. In addition, a comparison with the traditional Image Based Visual Servoing is presented. Real world experiments on a robot manipulator with the low cost vision system demonstrate the effectiveness of the proposed approach.