Flex-GP: genetic programming on the cloud

Authors:
Dylan Sherry;Kalyan Veeramachaneni;James McDermott;Una-May O'Reilly
Affiliations:
Massachusetts Institute of Technology;Massachusetts Institute of Technology;Massachusetts Institute of Technology;Massachusetts Institute of Technology
Venue:
EvoApplications'12 Proceedings of the 2012t European conference on Applications of Evolutionary Computation
Year:
2012

Citing 6
Cited 2

Spatially Structured Evolutionary Algorithms: Artificial Evolution in Space and Time (Natural Computing Series)

Spatially Structured Evolutionary Algorithms: Artificial Evolution in Space and Time (Natural Computing Series)
MapReduce: simplified data processing on large clusters

Communications of the ACM - 50th anniversary issue: 1958 - 2008
Empirical Validation of a Gossiping Communication Mechanism for Parallel EAs

Proceedings of the 2007 EvoWorkshops 2007 on EvoCoMnet, EvoFIN, EvoIASP,EvoINTERACTION, EvoMUSART, EvoSTOC and EvoTransLog: Applications of Evolutionary Computing
Order of nonlinearity as a complexity measure for models generated by symbolic regression via Pareto genetic programming

IEEE Transactions on Evolutionary Computation
Scaling Genetic Algorithms Using MapReduce

ISDA '09 Proceedings of the 2009 Ninth International Conference on Intelligent Systems Design and Applications
A Field Guide to Genetic Programming

A Field Guide to Genetic Programming

Lazy meta-learning: creating customized model ensembles on demand

WCCI'12 Proceedings of the 2012 World Congress conference on Advances in Computational Intelligence
Cloud driven design of a distributed genetic programming platform

EvoApplications'13 Proceedings of the 16th European conference on Applications of Evolutionary Computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

We describe Flex-GP, which we believe to be the first largescale genetic programming cloud computing system. We took advantage of existing software and selected a socket-based, client-server architecture and an island-based distribution model. We developed core components required for deployment on Amazon's EC2. Scaling the system to hundreds of nodes presented several unexpected challenges and required the development of software for automatically managing deployment, reporting, and error handling. The system's performance was evaluated on two metrics, performance and speed, on a difficult symbolic regression problem. Our largest successful Flex-GP runs reached 350 nodes and taught us valuable lessons for the next phase of scaling.