Reinforcement learning for fuzzy agents: application to a pighouse environment control

  • Authors:
  • Lionel Jouffe

  • Affiliations:
  • Parc Univ. Laval-Changé

  • Venue:
  • New learning paradigms in soft computing
  • Year:
  • 2002

Quantified Score

Hi-index 0.00

Visualization

Abstract

Fuzzy Actor-Critic Learning (FACL) and Fuzzy Q-learning (FQL) are reinforcement learning methods based on Dynamic Programming (DP) principles. In this chapter, they are used to tune on line the conclusion part of Fuzzy Inference Systems (FIS). The only information available for learning is the system feedback, which describes in terms of reward and punishment the task the fuzzy agent has to realize. At each time step, the agent receives a reinstate. The problem involves optimizing not only the direct reinforcement, but also the total amount of reinforcements the agent can receive in the future. To illustrate the use of these two learning methods, we first applied them to a problem in which we have to find a fuzzy controller to drive a boat from one bank to another, across a river with a strong non-linear current. Then, we used the well-known Cart-Pole Balancing and Mountain-Car problems to be able to compare our methods to other reinforcement learning methods, and focus on important characteristic aspects of FACL and FQL. The experimental studies had shown the superiority of these methods with respect to the other related methods we can find in the literature. We also found that our generic methods we can find in the literature. We also found that our generic methods allow us to learn every kind of reinforcement learning problem (continuous states, discrete/continuous actions, various types of reinforcement functions). Thanks to this flexibility, these learning methods have been applied successfully in an industrial problem, to discover a policy for pighouse environment control.