Co-Evolution in the Successful Learning of Backgammon Strategy

Authors:
Jordan B. Pollack;Alan D. Blair
Affiliations:
Computer Science Department, Volen Center for Complex Systems, Brandeis University, Waltham, MA 02254. E-mail: Email: pollack@cs.brandeis.edu;Computer Science Department, Volen Center for Complex Systems, Brandeis University, Waltham, MA 02254. E-mail: Email: blair@cs.uq.edu.au
Venue:
Machine Learning
Year:
1998

Citing 10
Cited 57

Connectionist learning of expert preferences by comparison training

Advances in neural information processing systems 1
Practical Issues in Temporal Difference Learning

Machine Learning
Toward an Ideal Trainer

Machine Learning
Temporal difference learning and TD-Gammon

Communications of the ACM
Massively parallel genetic programming

Advances in genetic programming
Learning to Predict by the Methods of Temporal Differences

Machine Learning
Tracking the Red Queen: Measurements of Adaptive Progress in Co-Evolutionary Simulations

Proceedings of the Third European Conference on Advances in Artificial Life
Competitive Environments Evolve Better Solutions for Complex Tasks

Proceedings of the 5th International Conference on Genetic Algorithms
Methods for Competitive Co-Evolution: Finding Opponents Worth Beating

Proceedings of the 6th International Conference on Genetic Algorithms
Algorithms for sequential decision-making

Algorithms for sequential decision-making

Comments on “Co-Evolution in the Successful Learning of Backgammon Strategy”

Machine Learning
Evolutionary techniques in physical robotics

Creative evolutionary systems
Programming backgammon using self-teaching neural nets

Artificial Intelligence - Chips challenging champions: games, computers and Artificial Intelligence
Three generations of automatically designed robots

Artificial Life
Learning to play strong poker

Machines that learn to play games
Asymmetric Co-evolution for Imperfect-Information Zero-Sum Games

ECML '00 Proceedings of the 11th European Conference on Machine Learning
First Three Generations of Evolved Robots

ER '01 Proceedings of the International Symposium on Evolutionary Robotics From Intelligent Robotics to Artificial Life
Co-evolving a Neural-Net Evaluation Function for Othello by Combining Genetic Algorithms and Reinforcement Learning

ICCS '01 Proceedings of the International Conference on Computational Science-Part II
Evolutionary Techniques in Physical Robotics

ICES '00 Proceedings of the Third International Conference on Evolvable Systems: From Biology to Hardware
A Game-Theoretic Approach to the Simple Coevolutionary Algorithm

PPSN VI Proceedings of the 6th International Conference on Parallel Problem Solving from Nature
Co-evolution, Determinism and Robustness

SEAL'98 Selected papers from the Second Asia-Pacific Conference on Simulated Evolution and Learning on Simulated Evolution and Learning
Automatic Symbolic Modelling of Co-evolutionarily Learned Robot Skills

IWANN '01 Proceedings of the 6th International Work-Conference on Artificial and Natural Neural Networks: Connectionist Models of Neurons, Learning Processes and Artificial Intelligence-Part I
Harmony Team

RoboCup 2000: Robot Soccer World Cup IV
Co-evolutionary Auction Mechanism Design: A Preliminary Report

AAMAS '02 Revised Papers from the Workshop on Agent Mediated Electronic Commerce on Agent-Mediated Electronic Commerce IV, Designing Mechanisms and Systems
Pareto Optimality in Coevolutionary Learning

ECAL '01 Proceedings of the 6th European Conference on Advances in Artificial Life
Beyond Samuel: evolving a nearly expert checkers player

Advances in evolutionary computing
Incremental training of first order recurrent neural networks to predict a context-sensitive language

Neural Networks
A Tournament-Based Competitive Coevolutionary Algorithm

Applied Intelligence
A multi-agent system integrating reinforcement learning, bidding and genetic algorithms

Web Intelligence and Agent Systems
SimEd: Simulating Education as a Multi Agent System

AAMAS '04 Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems - Volume 3
The MaxSolve algorithm for coevolution

GECCO '05 Proceedings of the 7th annual conference on Genetic and evolutionary computation
Multiagent simulation of learning environments

Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems
Cooperative Multi-Agent Learning: The State of the Art

Autonomous Agents and Multi-Agent Systems
Probabilistic neural network playing and learning Tic-Tac-Toe

Pattern Recognition Letters - Special issue: Artificial neural networks in pattern recognition
Evolutionary Body Building: Adaptive Physical Designs for Robots

Artificial Life
Ideal Evaluation from Coevolution

Evolutionary Computation
Combating Coevolutionary Disengagement by Reducing Parasite Virulence

Evolutionary Computation
The parallel Nash Memory for asymmetric games

Proceedings of the 8th annual conference on Genetic and evolutionary computation
Mindless Intelligence

IEEE Intelligent Systems
A Monotonic Archive for Pareto-Coevolution

Evolutionary Computation
Co-optimization algorithms

Proceedings of the 10th annual conference on Genetic and evolutionary computation
Evolving strategy for a probabilistic game of imperfect information using genetic programming

Genetic Programming and Evolvable Machines
Evolving Strategies for Non-player Characters in Unsteady Environments

EvoWorkshops '09 Proceedings of the EvoWorkshops 2009 on Applications of Evolutionary Computing: EvoCOMNET, EvoENVIRONMENT, EvoFIN, EvoGAMES, EvoHOT, EvoIASP, EvoINTERACTION, EvoMUSART, EvoNUM, EvoSTOC, EvoTRANSLOG
Evolving Teams of Cooperating Agents for Real-Time Strategy Game

EvoWorkshops '09 Proceedings of the EvoWorkshops 2009 on Applications of Evolutionary Computing: EvoCOMNET, EvoENVIRONMENT, EvoFIN, EvoGAMES, EvoHOT, EvoIASP, EvoINTERACTION, EvoMUSART, EvoNUM, EvoSTOC, EvoTRANSLOG
A co-evolutionary approach for military operational analysis

Proceedings of the first ACM/SIGEVO Summit on Genetic and Evolutionary Computation
Motivating Appropriate Challenges in a Reciprocal Tutoring System

Proceedings of the 2005 conference on Artificial Intelligence in Education: Supporting Learning through Intelligent and Socially Informed Technology
M2ICAL analyses HC-gammon

AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 1
A hybrid neural network and Minimax algorithm for zero-sum games

Proceedings of the 2009 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists
Neuroevolutionary Inventory Control in Multi-Echelon Systems

ADT '09 Proceedings of the 1st International Conference on Algorithmic Decision Theory
Coevolutionary temporal difference learning for Othello

CIG'09 Proceedings of the 5th international conference on Computational Intelligence and Games
No-limit texas hold'em poker agents created with evolutionary neural networks

CIG'09 Proceedings of the 5th international conference on Computational Intelligence and Games
Learning the ideal evaluation function

GECCO'03 Proceedings of the 2003 international conference on Genetic and evolutionary computation: PartI
Evolving parameterised policies for stochastic constraint programming

CP'09 Proceedings of the 15th international conference on Principles and practice of constraint programming
Critical factors in the empirical performance of temporal difference and evolutionary methods for reinforcement learning

Autonomous Agents and Multi-Agent Systems
Evolutionary mechanism design: a review

Autonomous Agents and Multi-Agent Systems
Learning n-tuple networks for othello by coevolutionary gradient search

Proceedings of the 13th annual conference on Genetic and evolutionary computation
Decision tree-based algorithms for implementing bot AI in UT2004

IWINAC'11 Proceedings of the 4th international conference on Interplay between natural and artificial computation - Volume Part I
Dynamic game difficulty balancing for backgammon

Proceedings of the 49th Annual Southeast Regional Conference
Evolving small-board Go players using coevolutionary temporal difference learning with archives

International Journal of Applied Mathematics and Computer Science
Immune based fuzzy agent plays checkers game

Applied Soft Computing
Adaptive reservoir computing through evolution and learning

Neurocomputing
A multimodal problem for competitive coevolution

AI'12 Proceedings of the 25th Australasian joint conference on Advances in Artificial Intelligence
Generating artificial neural networks for value function approximation in a domain requiring a shifting strategy

EvoApplications'13 Proceedings of the 16th European conference on Applications of Evolutionary Computation
Improving coevolution by random sampling

Proceedings of the 15th annual conference on Genetic and evolutionary computation
Quantitative analysis of the hall of fame coevolutionary archives

Proceedings of the 15th annual conference companion on Genetic and evolutionary computation
A survey on optimization metaheuristics

Information Sciences: an International Journal
An augmented EDA with dynamic diversity control and local neighborhood search for coevolution of optimal negotiation strategies

Applied Intelligence

Quantified Score

Hi-index	0.00

Visualization

Abstract

Following Tesauro‘s work on TD-Gammon, we used a 4,000 parameterfeedforward neural network to develop a competitive backgammonevaluation function. Play proceeds by a roll of the dice, applicationof the network to all legal moves, and selection of the position with the highest evaluation. However, no backpropagation,reinforcement or temporal difference learning methods were employed. Instead we applysimple hillclimbing in a relative fitness environment. We start withan initial champion of all zero weights and proceed simply by playingthe current champion network against a slightly mutated challenger andchanging weights if the challenger wins. Surprisingly, this workedrather well. We investigate how the peculiar dynamics of this domainenabled a previously discarded weak method to succeed, by preventingsuboptimal equilibria in a “meta-game” of self-learning.