neato 1.0.0

An implementation of NEAT (NeuroEvolution of Augmenting Topologies) by Kenneth O. Stanley

Neato

Neato is an implementation of NEAT (NeuroEvolution of Augmenting Topologies). It can be used to evolve simple neural networks to accomplish a specific task.

Contents

• Installation
• Getting Started
  • Genome
  • Ecosystem
• Examples

Installation

Neato is installed using pip on the command line.

pip install neato

Getting Started

Genome

Genomes are the evolvable neural networks that are the base of the NEAT algorithm. Each genome has three types of nodes: input, output, and hidden, and any number of connections between them. A genome starts out with only input and output nodes with no connections.

from neato import Genome

genome = Genome(2, 3)  # 2 input nodes and 3 output nodes

Once created, genomes can be passed numpy arrays to be evaluated by the network.

import numpy as np

x = np.array([0.5, 0.7])
y = genome(x)
print(y)

Out: [0. 0. 0.]

Since there are no connections, the output is all zeros. You can add connections manually or randomly through mutations. Once there are connections, the outputs are actual, non-zero values.

genome.add_connection(0, 2)     # Connect node 0 to node 2
genome.add_connection(1, 4)     # Connect node 1 to node 4
genome.mutate_add_connection()  # Connect two random nodes

y = genome(x)
print(y)

Out: [0.04911261 0.         0.34208322]

Nodes are numbered starting at 0 in the order that they are added to the genome. Since there are two inputs, 0 and 1 are inputs nodes and since there are three outputs, 2, 3, and 4 are output nodes. Hidden nodes can be added to the genome in the following manner:

genome.add_node(0)        # Add node on connection 0
genome.add_node(4)        # Add node on connection 4
genome.mutate_add_node()  # Add a node on a random connection

y = genome(x)
print(y)

Out: [0.01854924 0.         0.34208322]

Connections can now be added between the newly created nodes as shown below:

genome.add_connection(5, 6) # Connect hidden node 5 to hidden node 6

y = genome(x)
print(y)

Out: [0.01498559 0.         0.34208322]

Genomes can be evaluated as strings to see all nodes and connections.

print(genome)

Out: 

Ecosystem

An ecosystem is an environment that contains a population of genomes and manages them from generation to generation. Genomes within the ecosystem die, reproduce, mutate, and evolve.

from neato import Ecosystem

ecosystem = Ecosystem()

Once an ecosystem has been created, it need to be populated with genomes.

# Population size: 100 | The amount of genomes in the ecosystem
# Input size:      2   | The amount of input nodes for each genome
# Output size:     3   | The amount of output nodes for each genome

ecosystem.create_initial_population(100, input_size=2, output_size=3)

A population size will always remain constant because any genomes that are killed are replaced by new genomes. Genomes in a given ecosystem will always have the same amount of input and output nodes, but hidden nodes and connections will vary from genome to genome.

Another way to create a population is to supply a parent genome:

from neato import Genome

genome = Genome(2, 3)
ecosystem.create_initial_population(100, parent_genome=genome)

If a parent genome is given instead of input and output sizes, all genomes will be based on the parent. They will have the same amount of inputs and outputs as the parent but will be mutated slightly. Optionally, mutations can be bypassed by setting mutate to False as follows:

from neato import Genome

genome = Genome(2, 3)
ecosystem.create_initial_population(100, parent_genome=genome, mutate=False)

If an ecosystem is populated in this way, all genomes will be exact copies of the parent genome.

Once an population has been created, each genome can be evaluated and have a fitness value applied. Since each problem to be solved by NEAT is different, the fitness values must be applied manually by the user.

import random

for genome in ecosystem.get_population():
    x = np.array([0.5, 0.5])
    y = genome(x)
    genome.fitness = random.randrange(0, 100)  # Set fitness to random value

The fitness was applied randomly above, but it should be set as meaningful values depending on the desired result.

Once fitness values have been assigned, the ecosystem can move on to the next generation. Genomes with lower fitness values will be killed, and the survivors will reproduce to fill in their places. Mutations will be applied to the children in a similar manner to when the inital population was created.

ecosystem.next_generation()

The percentage of genomes that are killed with each generation can be adjusted with the kill_percentage parameter.

# Only kill 20% each generation
# Default is 50%
ecosystem.next_generation(kill_percentage=20)

Optionally, a parent genome can be specified which will cause the ecosystem to base the next generation upon. In this case, no other genomes will be crossed, but copies of the parent will be mutated.

# Get the genome with the highest fitness value
best_genome = ecosystem.get_best_genome()

# Base the next generation on this genome
ecosystem.next_generation(parent_genome=best_genome)

This process repeats for as long as required. The user determines how many generations to reach or to what fitness value must be achieved in order to stop.

Examples

The following example shows how Neato can be used to solve a simple Gym environment called CartPole. In this environment, a pole is attached to a cart and the agent must move the cart back and forth to balance it.

import gym

from neato import Ecosystem

POPULATION = 100
GENERATIONS = 10

# Set up environment
env = gym.make('CartPole-v1')

# Set up ecosystem
ecosystem = Ecosystem()
ecosystem.create_initial_population(POPULATION, input_size=4, output_size=2)

# Start the evolution
for generation in range(GENERATIONS):
    for genome in ecosystem.get_population():
        # Get the initial input from the environment        
        observation = env.reset()

        # Evaluate the genome until it is done
        # When it is done is determined by the environment
        done = False
        while not done:
            # Render the environment
            env.render()

            # Evaluate genome for given timestep
            action = genome(observation).argmax()

            # Evaluate reward and get next input
            observation, reward, done, info = env.step(action)

            # Apply the fitness to the genome
            genome.fitness += reward   

    # Display ecosystem info
    print(ecosystem)

    # Kill less fit genomes and cross fitter genomes
    ecosystem.next_generation()

# Close the environment
env.close()

For this and more examples, see the examples directory.