PopuLoRA (wip)
Implementation and explorations into PopuLoRA, Co-Evolving LLM Populations for Reasoning Self-Play, from Roger Castanyer et al at vmax.ai
Install
pip install populora
Usage
import torch
import torch.nn as nn
from populora import Population
# 2-layer MLP
model = nn.Sequential(
nn.Linear(2, 8),
nn.ReLU(),
nn.Linear(8, 1)
)
# wrap with Population
pop = Population(
model,
pop_size = 16,
low_rank = 4,
lora_targets = ['0', '2']
)
state = torch.randn(1, 4, 2)
# evaluate population against environment
preds = pop(state, all_individuals = True)
labels = torch.randn(1, 4, 1)
fitnesses = -((preds - labels ) ** 2).reshape(16, -1).mean(dim = -1)
# selection
result = pop.select(
selection_type = 'deterministic',
fitnesses = fitnesses,
survive_frac = 0.5
)
# parent selection
parents = pop.select_parents(
selection_type = 'tournament',
fitnesses = fitnesses,
num_children = len(result.selected_out_indices),
culled = result.selected_out_indices
)
# crossover
pop.crossover_('average', parents, result.selected_out_indices)
# mutate newly generated offspring, preserving surviving elite parents
pop.mutate_('full_gaussian', individuals = result.selected_out_indices)
# alternatively, mutate the entire population
pop.mutate_('full_gaussian', all_individuals = True)
# do the above in a for loop
# ...
# then pick the highest fitness individual and resume RL or fine-tuning on the base model
model = pop.select_and_merge_best_(fitnesses)
Distributed Evolution
Evolution parallelizes trivially - each rank evaluates its share of the population against the environment, the fitnesses are gathered, and the evolution step runs identically on every rank
The population is automatically moved to the distributed device (each rank's local GPU) on construction - pass device to Population to override
from time import sleep
import torch
from torch import nn
from populora import Population, is_main_rank
model = nn.Sequential(
nn.Linear(8, 16),
nn.ReLU(),
nn.Linear(16, 1)
)
pop = Population(
model,
pop_size = 16,
low_rank = 2,
lora_targets = ['0', '2']
)
x = torch.randn(1, 8)
def eval_env(population, idx):
sleep(0.1)
with torch.no_grad():
return population(x, individual = idx).abs().mean().item() + torch.randn(1).item()
for gen in range(10):
# distributed evaluation
fitnesses = pop.evaluate_distributed(eval_env)
if is_main_rank():
print(f'gen {gen:02d} | best: {fitnesses.max():.3f} | mean: {fitnesses.mean():.3f}')
# evolution step
pop.evolve_(fitnesses)
run on 4 processes
torchrun --standalone --nproc-per-node=4 evolve.py
or across machines
torchrun --nnodes=4 --nproc-per-node=1 --rdzv-endpoint=$MASTER_HOST:29500 evolve.py
Citations
@misc{castanyer2026populoracoevolvingllmpopulations,
title = {PopuLoRA: Co-Evolving LLM Populations for Reasoning Self-Play},
author = {Roger Creus Castanyer and Geoffrey Bradway and Lorenz Wolf and Maxwill Lin and Augustine N. Mavor-Parker and Matthew James Sargent},
year = {2026},
eprint = {2605.16727},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2605.16727},
}
@misc{schmidhuber2012powerplaytrainingincreasinglygeneral,
title = {POWERPLAY: Training an Increasingly General Problem Solver by Continually Searching for the Simplest Still Unsolvable Problem},
author = {Jürgen Schmidhuber},
year = {2012},
eprint = {1112.5309},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/1112.5309},
}
@misc{xu2026selfimprovinglanguagemodelsbidirectional,
title = {Self-Improving Language Models with Bidirectional Evolutionary Search},
author = {Guowei Xu and Zhenting Qi and Huangyuan Su and Weirui Ye and Himabindu Lakkaraju and Sham M. Kakade and Yilun Du},
year = {2026},
eprint = {2605.28814},
archivePrefix = {arXiv},
primaryClass = {cs.CL},
url = {https://arxiv.org/abs/2605.28814},
}
@misc{bahlousboldi2026vectorpolicyoptimizationtraining,
title = {Vector Policy Optimization: Training for Diversity Improves Test-Time Search},
author = {Ryan Bahlous-Boldi and Isha Puri and Idan Shenfeld and Akarsh Kumar and Mehul Damani and Sebastian Risi and Omar Khattab and Zhang-Wei Hong and Pulkit Agrawal},
year = {2026},
eprint = {2605.22817},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2605.22817},
}
@misc{bailey2026scalingselfplayselfguidance,
title = {Scaling Self-Play with Self-Guidance},
author = {Luke Bailey and Kaiyue Wen and Kefan Dong and Tatsunori Hashimoto and Tengyu Ma},
year = {2026},
eprint = {2604.20209},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2604.20209},
}
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
populora-0.1.9.tar.gz
(17.8 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
populora-0.1.9-py3-none-any.whl
(17.5 kB
view details)
File details
Details for the file populora-0.1.9.tar.gz.
File metadata
- Download URL: populora-0.1.9.tar.gz
- Upload date:
- Size: 17.8 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
uv/0.8.17
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
e9cfcbb4d354e6dbfd9014439db3927d1b518d442efede30f3b46711f3e88cf2
|
|
| MD5 |
05dbaa9e2d14def7994cb03f7251b6b3
|
|
| BLAKE2b-256 |
944604000dc37d72c43c34a009cc1a52f654262b3ef1e6c7413b13a8ad20936e
|
File details
Details for the file populora-0.1.9-py3-none-any.whl.
File metadata
- Download URL: populora-0.1.9-py3-none-any.whl
- Upload date:
- Size: 17.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
uv/0.8.17
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
635f360f2324e42ea0f0b6cf1252c7a5c0a35fc9f2b77abe14b879c2a2bcc56a
|
|
| MD5 |
ef9666e0e85f11a52a4a6cd7424afafb
|
|
| BLAKE2b-256 |
94b0b8e51a3c29000faa46380ced88e6174ba1519b207168caa79b612f59a875
|
