compel 0.1.5

A prompting enhancement library for transformers-type text embedding systems.

Compel

A text prompt weighting and blending library for transformers-type text embedding systems, by @damian0815.

With a flexible and intuitive syntax, you can re-weight different parts of a prompt string and thus re-weight the different parts of the embeddning tensor produced from the string.

Tested and developed against Hugging Face's StableDiffusionPipeline but it should work with any diffusers-based system that uses an Tokenizer and a Text Encoder of some kind.

Adapted from the InvokeAI prompting code (also by @damian0815). For now, the syntax is fully documented here - note however that cross-attention control .swap() is currently ignored by Compel.

Installation

pip install compel

Demo

see compel-demo.ipynb

Quickstart

with Hugging Face diffusers >=0.12:

from diffusers import StableDiffusionPipeline
from compel import Compel

pipeline = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5")
compel = Compel(tokenizer=pipeline.tokenizer, text_encoder=pipeline.text_encoder)

# upweight "ball"
prompt = "a cat playing with a ball++ in the forest"
conditioning = compel.build_conditioning_tensor(prompt)

# generate image
images = pipeline(prompt_embeds=conditioning, num_inference_steps=20).images
images[0].save("image.jpg")

For batched input, use torch.cat to merge multiple conditioning tensors into one:

import torch

from diffusers import StableDiffusionPipeline
from compel import Compel

pipeline = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5")
compel = Compel(tokenizer=pipeline.tokenizer, text_encoder=pipeline.text_encoder)

prompts = ["a cat playing with a ball++ in the forest", "a dog playing with a ball in the forest"]
prompt_embeds = torch.cat([compel.build_conditioning_tensor(prompt) for prompt in prompts])
images = pipeline(prompt_embeds=prompt_embeds).images

images[0].save("image0.jpg")
images[1].save("image1.jpg")