A library that makes Evolutionary Strategies (ES) simple to use.

# EvoStrat

A library that makes Evolutionary Strategies (ES) simple to use.

### Installation

pip install evostrat

### Usage

pop = PopulationImpl(...) # See complete examples for implementations.
optim = torch.optim.Adam(pop.parameters()) # Use any torch.optim optimizer
for i in range(N):
optim.step()


For complete examples that solves 'LunarLander-v2' see the examples/.

### Description

Evolutionary Strategies is a powerful approach to solve reinforcement learning problems and other optimization problems where the gradients cannot be computed with backprop. See "Evolution strategies as a scalable alternative to reinforcement learning" for an excellent introduction.

In ES the objective is to maximize the expected fitness of a distribution over individuals, referred to as the population. With a few math tricks this objective can be maximized with gradient ascent, even if the fitness function itself is not differentiable.

This library offers

1. A flexible and natural interface for ES that cleanly separates the environment, the reinforcement learning agent, the population distribution and the optimization.
2. A plug-and-play approach for reinforcement learning agents with torch.nn.Module policy networks. See examples/lunar_lander.py and examples/normal_lunar_lander.py.
3. Several population distributions and variants
1. Independent Normal. equivalent to OpenAI ES or PEPG depending on whether the standard deviation is fixed or learned. See examples/normal_lunar_lander.py
2. Multivariate Normal with a full covariance matrix. Similar to CMA-ES. See examples/multivariate_normal_lunar_lander.py
3. Categorical. For agents with categorical parameters, demonstrating the ability to handle non-normal distributions. See the examples/binary_lunar_lander.py.
4. A simple interface for creating your own populations, without having to derive any gradients! Just subclass Population and implement the sampling process. See the built in populations for inspiration.

@misc{palm2020,
author = {Palm, Rasmus Berg},
title = {EvoStrat},
year = {2020},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/rasmusbergpalm/evostrat}}
}


## Project details

Uploaded source
Uploaded py3