Skip to main content

x-evolution

Implementation of various evolutionary algorithms, starting with evolutionary strategies

explained by @bycloud

Install

$ pip install x-evolution

Usage

import torch
from x_evolution import EvoStrategy

# model

from torch import nn
model = torch.nn.Sequential(
    nn.Linear(8, 16),
    nn.ReLU(),
    nn.Linear(16, 4)
)

# evolution wrapper

evo_strat = EvoStrategy(
    model,
    environment = lambda model: torch.randint(0, 100, ()), # environment is just a function that takes in the individual model (with unique noise) and outputs a scalar (the fitness) the measure you are selecting for
    noise_population_size = 30, # increase this for better gradient estimates
    noise_scale = 1e-2,         # the scale of the perturbation noise, also the initial noise scale (sigma) if `learned_noise_scale` = True
    num_generations = 100,    # number of generations / training steps
    learning_rate = 1e-3,     # scale on update derived by fitness and perturb noises
    params_to_optimize = None, # defaults to all parameters, but can be [str {param name}] or [Parameter]
    noise_dtype = None        # torch.float16 / torch.bfloat16 / torch.int8 for quantized noise - https://arxiv.org/abs/2602.03120
)

# do evolution with your desired fitness function for so many generations

evo_strat()

# model will be saved under checkpoints/ folder
# can also specify checkpoint_every at init and select the one with your favored fitness score for continued policy gradient learning etc

Preventing Catastrophic Forgetting (Anchored Weight Decay)

Continuing from a pretrained checkpoint, ES random-walks in directions the new task barely constrains, which can erode prior capabilities. Anchored Weight Decay (AWD) from Schweighofer et al. (2026) pulls weights toward their init snapshot without changing the optimizer. This is the L2 form, equivalent to L2 Init (Kumar et al., 2023). Weights are snapshotted at construction:

evo_strat = EvoStrategy(
    pretrained_model,
    environment = environment,
    num_generations = 100,
    noise_population_size = 32,
    learning_rate = 3e-3,
    anchored_weight_decay_rate = 10.   # L2 AWD penalty factor (paper's lambda); lr * rate is pulled toward the anchor each generation
)

evo_strat()

# after interleaving with gradient-based training (e.g. PPO -> ES -> PPO -> ES), re-anchor to the latest weights
evo_strat.reanchor_()

# or anchor to an explicit checkpoint / module
evo_strat.set_init_anchor_(torch.load('earlier.pt'))

anchored_weight_decay_rate is optimizer agnostic and also works with use_optimizer = False and custom optimizers. In EvoStrategy the pull per step is learning_rate * anchored_weight_decay_rate; CMAEvoStrategy applies the rate directly once per base update, so it requires freeze_base = False or merge_every. When using the default SGD optimizer, a positive rate turns off zero-targeting weight decay.

CMA-ES

CMAEvoStrategy implements full-covariance CMA-ES over a low-rank adapter, so an (m, n) weight costs r * (m + n) dimensions instead of m * n:

from x_evolution import CMAEvoStrategy

cma = CMAEvoStrategy(
    model,
    environment = environment,
    num_generations = 100,
    lora_rank = 1,          # None for the full weight update
    freeze_base = True,     # False recenters the base every generation
    noise_scale = 1e-2,     # std of the induced weight delta
    noise_dtype = None      # torch.float16 / torch.bfloat16 / torch.int8 for quantized noise - https://arxiv.org/abs/2602.03120
)

cma()

Pass sep_cma = True for diagonal covariance. params_to_optimize accepts the same scopes as EvoStrategy; pass scope = 'name' to update one scope at a time. 1D parameters such as biases are skipped.

Set merge_every = N with freeze_base = True to fold the current residual into the base every N generations. Mirror sampling is skipped automatically on the first generation after a merge.

train_lunar.py switches between the two with --algorithm es or --algorithm cma.

Distributed

Using the CLI from 🤗

$ uv run accelerate config

Then

$ uv run accelerate launch train_lunar.py

Or with torchrun (which can do multiple CPU processes for testing)

$ uv run torchrun --nproc_per_node=<N> train_lunar.py --cpu True

For a quick demo on LunarLander with 8 processes

$ uv run torchrun --nproc_per_node=8 train_lunar.py --cpu True --noise_population_size 100 --fitness_to_weighted_factor centered_rank

Sim-to-Real Bipedal Locomotion (Microduck)

Train a command-conditioned walking gait for the Microduck biped robot from scratch using EvoStrategy and torchrun. The four driven leg joints follow an inline CPG; the policy controls swing, knee scale, phase offsets, and the remaining joint targets.

With 8 CPU processes, a stable gait typically emerges within a few hundred generations:

$ uv run torchrun --nproc_per_node=8 train_microduck.py

Training samples random linear and angular velocity commands by default. Evaluations use --eval_command (default 0.15,0); rendered videos show a 3D arrow above the duck's head pointing along the commanded motion. Use --horizon 800 for 16-second episodes.

To render evaluation videos or resume from the saved best checkpoint:

# render periodic evaluation videos to videos/
$ uv run torchrun --nproc_per_node=8 train_microduck.py --eval_video True --eval_every 50

# resume training from best checkpoint
$ uv run torchrun --nproc_per_node=8 train_microduck.py --resume True

Special thanks to Pollen Robotics 🤗 for their open sourcing of such a polished sim2real env!

Citations

@article{xu2026quantized,
    title   = {Quantized Evolution Strategies: High-precision Fine-tuning of Quantized LLMs at Low-precision Cost},
    author  = {Yinggan Xu and Kajetan Schweighofer and Risto Miikkulainen and Xin Qiu},
    journal = {ArXiv},
    year    = {2026},
    volume  = {abs/2602.03120},
    url     = {https://arxiv.org/abs/2602.03120}
}
@article{Qiu2025EvolutionSA,
    title   = {Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning},
    author  = {Xin Qiu and Yulu Gan and Conor F. Hayes and Qiyao Liang and Elliot Meyerson and Babak Hodjat and Risto Miikkulainen},
    journal = {ArXiv},
    year    = {2025},
    volume  = {abs/2509.24372},
    url     = {https://api.semanticscholar.org/CorpusID:281674745}
}
@misc{sarkar2025evolutionstrategieshyperscale,
    title   = {Evolution Strategies at the Hyperscale},
    author  = {Bidipta Sarkar and Mattie Fellows and Juan Agustin Duque and Alistair Letcher and Antonio León Villares and Anya Sims and Dylan Cope and Jarek Liesen and Lukas Seier and Theo Wolf and Uljad Berdica and Alexander David Goldie and Aaron Courville and Karin Sevegnani and Shimon Whiteson and Jakob Nicolaus Foerster},
    year    = {2025},
    eprint  = {2511.16652},
    archivePrefix = {arXiv},
    primaryClass = {cs.LG},
    url     = {https://arxiv.org/abs/2511.16652},
}
@misc{fortunato2019noisynetworksexploration,
    title   = {Noisy Networks for Exploration},
    author  = {Meire Fortunato and Mohammad Gheshlaghi Azar and Bilal Piot and Jacob Menick and Ian Osband and Alex Graves and Vlad Mnih and Remi Munos and Demis Hassabis and Olivier Pietquin and Charles Blundell and Shane Legg},
    year    = {2019},
    eprint  = {1706.10295},
    archivePrefix = {arXiv},
    primaryClass = {cs.LG},
    url     = {https://arxiv.org/abs/1706.10295},
}
@article{ha2017visual,
    title   = "A Visual Guide to Evolution Strategies",
    author  = "Ha, David",
    journal = "blog.otoro.net",
    year    = "2017",
    url     = "https://blog.otoro.net/2017/10/29/visual-evolution-strategies/"
}
@misc{dahal2026backprop,
    title  = {Dust: Pretraining Transformers Without Backpropagation},
    author = {Dahal, Samip and Mandal, Bishwas and G{\"u}lbahar, Serdar and Vegesna, Akshay},
    year   = {2026},
    url    = {https://qlabs.sh/research/dust}
}
@article{hansen2001completely,
    title   = {Completely Derandomized Self-Adaptation in Evolution Strategies},
    author  = {Hansen, Nikolaus and Ostermeier, Andreas},
    journal = {Evolutionary Computation},
    year    = {2001},
    volume  = {9},
    number  = {2},
    pages   = {159--195},
    doi     = {10.1162/106365601750190398}
}
@inproceedings{ros2008simple,
    title     = {A Simple Modification in {CMA-ES} Achieving Linear Time and Space Complexity},
    author    = {Ros, Raymond and Hansen, Nikolaus},
    booktitle = {Parallel Problem Solving from Nature ({PPSN X})},
    series    = {Lecture Notes in Computer Science},
    volume    = {5199},
    pages     = {296--305},
    year      = {2008},
    doi       = {10.1007/978-3-540-87700-4_30}
}
@inproceedings{akimoto2014comparison,
    title     = {Comparison-Based Natural Gradient Optimization in High Dimension},
    author    = {Akimoto, Youhei and Auger, Anne and Hansen, Nikolaus},
    booktitle = {Proceedings of the Genetic and Evolutionary Computation Conference ({GECCO})},
    pages     = {373--380},
    year      = {2014},
    doi       = {10.1145/2576768.2598258}
}
@article{schweighofer2026overcoming,
    title   = {Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies},
    author  = {Kajetan Schweighofer and Conor F. Hayes and Roberto Dailey and Risto Miikkulainen and Xin Qiu},
    journal = {ArXiv},
    year    = {2026},
    volume  = {abs/2605.30148},
    url     = {https://arxiv.org/abs/2605.30148}
}
@article{kumar2023maintaining,
    title   = {Maintaining Plasticity in Continual Learning via Regenerative Regularization},
    author  = {Saurabh Kumar and Henrik Marklund and Benjamin Van Roy},
    journal = {ArXiv},
    year    = {2023},
    url     = {https://arxiv.org/abs/2308.11958}
}
@inproceedings{anonymous2026continual,
    title   = {Continual Reinforcement Learning with Neuroevolution},
    author  = {Anonymous},
    booktitle = {Submitted to The Fifteenth International Conference on Learning Representations},
    year    = {2026},
    url     = {https://openreview.net/forum?id=UxsGk4on0e},
    note    = {under review}
}

Nothing makes sense except in the light of evolution - Theodosius Dobzhansky

Metadata

Release files for x-evolution 0.2.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for x-evolution 0.2.3
File Size Uploaded Provenance Actions
x_evolution-0.2.3.tar.gz 26.8 kB Without provenance Details for x_evolution-0.2.3.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for x-evolution 0.2.3
File Interpreter ABI Platform Actions
x_evolution-0.2.3-py3-none-any.whl Python 3 none any Details for x_evolution-0.2.3-py3-none-any.whl

Total release size: 50.7 kB

Release files / x_evolution-0.2.3.tar.gz

Download URL x_evolution-0.2.3.tar.gz
Size 26.8 kB
Tags Source
SHA-256 checksum
How to use checksums
3a6575ec4240a78ab9cbc5beab30bbc00837978997ad66753c5e60512740b4c9
BLAKE2b-256 checksum
How to use checksums
47c1037f8cd1b93046e756793f79e312f61a92a778c6ef4b94e7b1742ac07012
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.17

Release files / x_evolution-0.2.3-py3-none-any.whl

Download URL x_evolution-0.2.3-py3-none-any.whl
Size 23.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
7c8e0272b441fb8d9e498acb998bc9eb9d08b9e27d79bc3a609953da928f8144
BLAKE2b-256 checksum
How to use checksums
d17c1d27d9d511e97341a0d8ab0a485d771a4f33f92c92b0efda9af83568e8ae
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.17

Security for x-evolution 0.2.3

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.2.3 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page