latent-spaces-by-example 0.1.2

Latent spaces by example.

latent-spaces-by-example

Python package: latent-spaces-by-example (import as latent_spaces_by_example).

What this repo is

This repo provides utilities for performing black-box search/optimisation over the outputs of generative (diffusion/flow matching) models.

This works by defining surrogate latent spaces that map a model’s high-dimensional latent representation into a low-dimensional space where standard black-box optimisation algorithms can be applied.

You form these spaces by providing a small set of latents corresponding to specific outputs of the generative model (called “seeds”). A surrogate latent space then defines maps between:

• u: hypercube coordinates, shape ((N, n_seeds-1))
• w: nonnegative weights on the unit sphere, shape ((N, n_seeds))
• z: latent vectors, shape ((N, n_latent_dims))

The main API is latent_spaces_by_example.SurrogateChart.

Paper

Defining latent spaces by example: optimisation over the outputs of generative models

Install

Create / sync the environment (uses uv):

uv sync

Quick usage

The minimal workflow is:

• sample seed_latents (shape (num_seeds, latent_dim)) from the latent distribution
• build a chart (e.g. knothe_rosenblatt_surrogate_chart(seed_latents))
• pick a coordinate u \in [0,1]^(num_seeds-1))
• map u -> z and optionally round-trip z -> u
• generate z -> x using the generative model

See examples/basic_surrogate_chart.py.

Examples

Flux + CLIP optimisation (3 seeds → 2D u-space)

This example optimises a 2D coordinate (u \in [0,1]^2) (formed from three seed latents) to maximise a CLIP similarity score of images generated by Flux. It also evaluates a 2D grid and saves a contour plot. In this example the basis latents are sampled randomly (i.e. they correspond to random images).

Install optional dependencies:

uv sync --group examples

Dry run (no model download; checks shape plumbing only):

uv run python examples/image_optimisation.py --dry-run

Small real run (downloads models; will be slow and may require GPU + huggingface-cli login):

uv run python examples/image_optimisation.py

Development

Run tests

uv run pytest

Lint

uv run ruff check .

Format

uv run isort --profile black . && black .