Skip to main content

Evolution Through Large Models

Project description

DOI

OpenELM

OpenELM is an open-source library by CarperAI, designed to enable evolutionary search with language models in both code and natural language.

The OpenELM project has the following goals:

  1. Release an open-source version of ELM with its associated diff models.
  2. Integrate with both open-source language models (run locally or on Colab) and with closed models via paid APIs, such as the OpenAI API. We want to support users with many different compute profiles!
  3. Provide a simple interface to a range of example environments for evolutionary search, to let users adapt these easily for their domain.
  4. Demonstrate the potential of evolution with LLMs.

Install

pip install openelm

To use the sodarace environment, you must first pip install swig.

Then:

pip install openelm[sodaracer]

See the pyproject.toml for further install options.

Features

LLM integration with evolutionary algorithms

OpenELM supports the quality-diversity algorithms MAP-Elites, CVT-MAP-Elites, and Deep Grid MAP-Elites, as well as a simple genetic algorithm baseline.

Evolutionary operators

OpenELM supports:

  1. Prompt-based mutation with instruct models
  2. Diff models (specialised for code)
  3. Crossover with language models

LLM support, efficiency, and safety

OpenELM’s language models are instantiated as Langchain classes by default, which means that OpenELM can support practically any existing LLM API, as well as models run on your local GPU via HuggingFace Transformers.

We also provide optional Nvidia Triton Inference Server support, intended for use cases where low latency on 8 or more GPUs is important. Finally, for code generation domains, we provide a sandbox environment, consisting of a container server backed with gVisor (a container runtime that introduces an additional barrier between the host and the container) as well as a heuristic-based safety guard.

Baseline environments

  1. Sodarace. A 2D physics-based simulation of robots moving across a variety of terrains. These robots are created by Python programs generated from an LLM.
  2. Image Generation. OpenELM can evolve over generated images by generating code that returns NumPy arrays containing the images. This serves as a simple test environment for code generation
  3. Programming Puzzles. OpenELM can be used to generate diverse solutions to programming puzzles. This environment supports co-evolution of both the problem and the solution at the same time.
  4. Prompts. OpenELM contains a generic environment suitable for evolving prompts for language models, customizable with Langchain templates to the desired domain.
  5. We also include a poetry environment, demonstrating the use of LLMs to evaluate both the quality and diversity of generated creative writing text, as described in a recent CarperAI blog post on Quality-Diversity with AI Feedback (QDAIF).

Architecture

Roughly, ELM consists of a pipeline of different components:

  1. The Environment class. This class defines the mechanics of how to initialize members of the population, mutate them with the desired operator, and how to measure the fitness (and diversity) of individuals.
  2. The MAPElites class. This class describes how the evolutionary algorithm works, and can be viewed as a wrapper around the environment defining the selection algorithm for generated individuals.
  3. The MutationModel class, which is responsible for running the LLM to actually generate new individuals. This functions as a wrapper around the LangChain API. The environment is expected to call the MutationModel when a new individual is needed.
  4. The ELM class calls the MAPElites algorithm class and runs the search.

All options for these classes are defined in configs.py, via dataclasses which are registered as a hydra config, and can be overriden via the command line when running one of the example scripts such as run_elm.py.

Running ELM

python run_elm.py will start an ELM evolutionary search using the defaults listed in configs.py. These can be overriden via the command line. For example, you can use run_elm.py env=image_evolution to run the Image Evolution environment.

Sandbox

To use the code execution sandbox, see the sandboxing readme for instructions to set it up in a Docker container with the gVisor runtime.

Triton

We also have code available to run models in Nvidia's Triton Inference Server. See the Triton Readme to get started

Contributing

If you'd like to contribute or have questions, go to the #openelm channel on the CarperAI discord!

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

OpenELM-0.9.2.tar.gz (3.9 MB view details)

Uploaded Source

Built Distribution

OpenELM-0.9.2-py3-none-any.whl (2.5 MB view details)

Uploaded Python 3

File details

Details for the file OpenELM-0.9.2.tar.gz.

File metadata

  • Download URL: OpenELM-0.9.2.tar.gz
  • Upload date:
  • Size: 3.9 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.9.16

File hashes

Hashes for OpenELM-0.9.2.tar.gz
Algorithm Hash digest
SHA256 cb7bdece50d131cb829e5bc6c2db70250887f9f310c486e7f47827990ccfaacf
MD5 33cb5f6453af10e3817c68f854fdc37e
BLAKE2b-256 2032c86ce924bdfeb2cd6e542cad9d9cd7e5855e3ffd4bbf0b628d38cf5b7fde

See more details on using hashes here.

File details

Details for the file OpenELM-0.9.2-py3-none-any.whl.

File metadata

  • Download URL: OpenELM-0.9.2-py3-none-any.whl
  • Upload date:
  • Size: 2.5 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.9.16

File hashes

Hashes for OpenELM-0.9.2-py3-none-any.whl
Algorithm Hash digest
SHA256 429552c37cdbe3a75dee3d8416caa5e801073a0914404fb3c8c8dfd2fd397ba6
MD5 4583dec6744872f140adbba59a4e6df4
BLAKE2b-256 5a74dd98318b688e049470c531e25bc48b3e1fb0ef2bd63f10fff4f74785ed20

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page