genji 1.0.0

Jinja2-based templating for LLM-generated structured output

Genji

Genji is a templating library for LLM-generated structured output, built on Jinja2. It ensures templates own the structure and syntax (JSON brackets, HTML tags, YAML indentation) while LLMs only generate content, guaranteeing valid output every time.

The problem: LLMs often produce malformed JSON, broken HTML, or invalid YAML when asked to generate structured output directly.

The solution: Separate concerns. Templates define structure, LLMs fill in content. Structure is guaranteed, content is generated.

Installation

From PyPI (Recommended)

# With uv (recommended)
uv pip install genji

# With pip
pip install genji

From Source

git clone https://github.com/calebevans/genji.git
cd genji
uv pip install -e .

For development:

uv pip install -e ".[dev]"
pre-commit install

Quick Start

from genji import Template, LLMBackend

# Configure the LLM backend
backend = LLMBackend(model="gpt-4o-mini")

# Define a template (default_filter="json" applies to all gen() calls)
template = Template("""
{
  "greeting": {{ gen("a friendly greeting for {name}") }},
  "farewell": {{ gen("a warm farewell for {name}") }}
}
""", backend=backend, default_filter="json")

# Render with variables
result = template.render(name="Alice")
print(result)  # Valid JSON guaranteed

# Or parse directly to dict
data = template.render_json(name="Alice")
print(data["greeting"])  # LLM-generated greeting

Note: On first run, LiteLLM may download model configurations. Subsequent runs use cached data.

Features

Template Syntax

Genji extends Jinja2 with a gen() function for LLM generation:

# Basic generation
{{ gen("a creative tagline") }}

# With variable interpolation
{{ gen("a description of {product}") }}

# With generation parameters
{{ gen("a tweet", max_tokens=280, temperature=0.9) }}

# With filters for different formats
{{ gen("content") | json }}   # JSON-safe string with quotes
{{ gen("content") | html }}   # HTML entity escaping
{{ gen("content") | yaml }}   # YAML-safe string
{{ gen("content") | xml }}    # XML entity escaping

All standard Jinja2 features are supported:

{# Comments #}

{% if condition %}
  {{ gen("something") }}
{% endif %}

{% for item in items %}
  {{ gen("content for {item}") }}
{% endfor %}

Format-Specific Filters

Genji provides filters for safe escaping in different formats:

Filter	Purpose	Example Output
json	JSON string with quotes	"Hello \"World\""
html	HTML entity escaping	<b>text</b>
xml	XML entity escaping	<tag>content</tag>
yaml	YAML-safe string	"key: value"
raw	No escaping (use carefully!)	<dangerous>
strip	Remove whitespace	"text"
lower	Lowercase	"hello"
upper	Uppercase	"HELLO"
truncate(n)	Truncate to n chars	"Long te..."

Important: The json filter outputs a complete JSON string value including quotes:

{{ gen("text") | json }}  # Outputs: "the generated text"

Default Filters

Avoid repetition by setting a default filter:

# Apply | json to all gen() calls automatically
template = Template(source, backend, default_filter="json")

# Or use file extension auto-detection
template = Template.from_file("report.json.genji", backend)
# Auto-detects "json" filter from .json.genji extension

# Override for specific prompts when needed
{{ gen("normal content") }}      # Gets json filter
{{ gen("special") | raw }}       # Skips filter
{{ gen("html content") | html }} # Uses html instead

LLM Backend Support

Genji uses LiteLLM for unified access to 100+ LLM providers.

See the full list of supported models.

OpenAI

backend = LLMBackend(
    model="gpt-4o-mini",
    api_key="sk-...",  # pragma: allowlist secret
)

Anthropic Claude

backend = LLMBackend(
    model="claude-3-5-sonnet-20241022",
    api_key="sk-ant-...",  # pragma: allowlist secret
)

Google Gemini

backend = LLMBackend(
    model="gemini/gemini-2.5-flash",
    api_key="...",  # pragma: allowlist secret
)

Local Ollama

backend = LLMBackend(
    model="ollama/llama3",
    base_url="http://localhost:11434"
)

Azure OpenAI

backend = LLMBackend(
    model="azure/your-deployment-name",
    api_key="...",  # pragma: allowlist secret
    base_url="https://your-resource.openai.azure.com"
)

For a complete list of supported models, see LiteLLM's model documentation.

Loading Templates from Files

# Create a template file: templates/report.json.genji
template = Template.from_file("templates/report.json.genji", backend)
result = template.render(topic="climate change")

Batch Generation

Genji automatically batches multiple gen() calls for efficiency:

template = Template("""
{
  "field1": {{ gen("prompt1") | json }},
  "field2": {{ gen("prompt2") | json }},
  "field3": {{ gen("prompt3") | json }}
}
""", backend=backend)

# All 3 prompts are sent to the LLM in parallel!
result = template.render()

Async Support

Every synchronous method has an async counterpart prefixed with a. This works with any async framework built on asyncio (FastAPI, aiohttp, etc.):

import asyncio
from genji import Template, LLMBackend

backend = LLMBackend(model="gpt-4o-mini")
template = Template("""
{
  "greeting": {{ gen("a friendly greeting for {name}") }},
  "farewell": {{ gen("a warm farewell for {name}") }}
}
""", backend=backend, default_filter="json")

async def main():
    # Async rendering
    result = await template.arender(name="Alice")

    # Async render + JSON parse
    data = await template.arender_json(name="Alice")

    # Async file loading
    t = await Template.afrom_file(
        "report.json.genji", backend
    )

asyncio.run(main())

When the backend supports native async (as LLMBackend does via litellm.acompletion), all LLM calls use true async I/O. If a backend only implements the sync protocol, arender automatically falls back to running the sync calls in a thread via asyncio.to_thread.

The synchronous API (render, render_json, from_file) is unchanged and continues to work exactly as before.

API Reference

Template

class Template:
    def __init__(
        self,
        source: str,
        backend: LLMBackend | MockBackend,
        default_filter: str | None = None
    ) -> None:
        """Initialize a template from a string.

        Args:
            source: Template string with Jinja2 syntax and gen() calls.
            backend: LLM backend instance (LLMBackend or MockBackend).
            default_filter: Optional default filter to apply to all gen() calls
                (e.g., "json", "html", "yaml"). Can be overridden per-prompt.
        """

    @classmethod
    def from_file(
        cls,
        path: str | Path,
        backend: LLMBackend | MockBackend,
        default_filter: str | None = None
    ) -> Template:
        """Load a template from a file.

        Args:
            path: Path to template file.
            backend: LLM backend instance.
            default_filter: Optional default filter. If None, auto-detects from
                file extension (.json.genji -> "json", .html.genji -> "html", etc.).
        """

    def render(self, **context: Any) -> str:
        """Render the template with the given context variables.

        Returns:
            Rendered template as a string.
        """

    def render_json(self, **context: Any) -> dict[str, Any]:
        """Render the template and parse as JSON.

        Returns:
            Parsed JSON as a Python dict.

        Raises:
            TemplateRenderError: If output is not valid JSON.
        """

    async def arender(self, **context: Any) -> str:
        """Async version of render()."""

    async def arender_json(self, **context: Any) -> dict[str, Any]:
        """Async version of render_json()."""

    @classmethod
    async def afrom_file(
        cls,
        path: str | Path,
        backend: LLMBackend | MockBackend,
        default_filter: str | None = None
    ) -> Template:
        """Async version of from_file()."""

LLMBackend

class LLMBackend:
    def __init__(
        self,
        model: str | None = None,
        api_key: str | None = None,
        base_url: str | None = None,
        temperature: float | None = None,
        max_tokens: int | None = None,
        add_system_prompt: bool = True,
        **kwargs: Any,
    ) -> None:
        """Initialize the LiteLLM backend.

        Args:
            model: Model name (required, or set GENJI_MODEL env var).
            api_key: API key (or set via environment variable).
            base_url: Base URL for custom endpoints.
            temperature: Temperature for generation (None uses provider default).
            max_tokens: Max tokens per generation (None uses provider default).
            add_system_prompt: Whether to add instruction for concise responses.
                Defaults to True.
            **kwargs: Additional arguments passed to litellm.completion().
        """

Per-Prompt Parameters

You can configure generation parameters for individual gen() calls:

# Control tokens, temperature, and stop sequences per prompt
{{ gen("short title", max_tokens=20) }}
{{ gen("creative content", temperature=0.9) }}
{{ gen("haiku", stop=["\n\n"]) }}

Smart Prompting

By default, Genji adds a system instruction to ensure LLMs return literal, concise responses:

# Default - LLM returns exactly what's requested
backend = LLMBackend(model="gpt-4o-mini")
# "a title" returns one title, not a list of options

# Disable for full control
backend = LLMBackend(model="gpt-4o-mini", add_system_prompt=False)

MockBackend

For testing without API calls:

from genji import MockBackend

backend = MockBackend(default_response="Test content")
# or
backend = MockBackend(response_fn=lambda prompt: f"Response to: {prompt}")

Configuration

Parameter	Default	Environment Variable	Description
model	Required	GENJI_MODEL	LLM model name (must be specified)
api_key	None	GENJI_API_KEY	API key for provider
base_url	None	GENJI_BASE_URL	Custom endpoint URL
temperature	Provider default	N/A	Temperature for generation
max_tokens	Provider default	N/A	Max tokens per generation
add_system_prompt	True	N/A	Add conciseness instruction

Error Handling

Genji provides clear exception types:

• GenjiError - Base exception
• TemplateParseError - Invalid template syntax
• TemplateRenderError - Error during rendering
• BackendError - LLM backend failure
• FilterError - Filter application failure