chainfactory-py 0.0.13b0

Structured LLM Inference with Easy Parallelism

ChainFactory: Run Structured LLM Inference with Easy Parallelism (chainfactory-py 0.0.13)

Introduction

ChainFactory is a powerful system for creating complex, multi-step LLM workflows using a simple YAML-like syntax. It allows you to connect multiple prompts in a chain, with outputs from one step feeding into inputs of subsequent steps. The most important feature is the reduced reliance on exact wording of the prompts and easy parallel execution in intermediate steps.

Key Features

• Sequential and parallel chain execution
• Automatic prompt generation from purpose statements
• Type-safe outputs with Pydantic models
• Chain inheritance and reusability
• Smart input/output mapping between chain links
• Hash based caching for prompts generated from purpose statements and masks for convex transitions.

Basic Concepts

Chain Links

A chain link is a single unit in your workflow, defined using the @chainlink directive:

@chainlink my-first-chain
prompt: Write a story about {topic}
out:
    story: str

Sequential vs Parallel Execution

You can specify how chain links execute:

• Sequential (-- or sequential): Links run one after another
• Parallel (|| or parallel): Links run simultaneously for multiple inputs

Example - a 3 step chain:

@chainlink generator --     # runs once
@chainlink reviewer ||      # runs multiple times in parallel, number of runs is determined by output of the previous link
@chainlink summarizer --    # runs once to summarize the output of the previous parallel link

Cool Features

1. Purpose-Driven Prompts

Instead of writing prompts manually, let ChainFactory generate them:

@chainlink
purpose: generate creative haiku topics
in:
    num: int
out:
    topics: list[str]

The system will automatically create an optimal prompt based on the purpose and the input variales before executing the chain.

2. Chain Inheritance

Reuse existing chains with @extends:

@extends examples/base_chain.fctr
@chainlink additional_step

3. Smart Input/Output Mapping

The system automatically maps outputs to inputs between chain links using dot notation:

in:
  previous_chain.element.field: str

4. Type Safety

Define your output structures:

def:
  Haiku:
    text: str
    explanation: str?    # optional field
out:
  haikus: list[Haiku]

Real-World Examples

1. Haiku Generator and Reviewer

@chainlink haiku-generator
prompt: Write {num} haiku(s) about {topic}
out:
    haikus: list[Haiku]

@chainlink reviewer ||     # parallely review each haiku
purpose: critically analyze each haiku

2. Weird Snack Combo Generator + Parallel Filter + Email Writera

@chainlink generate --
purpose: Generate {num} combinations of snacks that go well with each other. Generate {num} such combinations.
def:
  SnackCombo:
    items: list[str]
    comment: str?
out:
  combos: list[Combo]

@chainlink comment || # runs len(combos) instances of the chainlink
purpose: Given a snack combination, sarcastically comment on why it's the weirdest snack combination ever.
in:
  combos.element.items: str
out:
  res: list[SnackCombo]

Best Practices

1. Use Purpose Statements When possible, let the system generate prompts using clear one-liner purpose statements.

2. Type Everything Define input/output types for better reliability:

def:
  MyType:
    field1: str
    field2: int?

3. Chain Structure - General Workflows

• Start with sequential chains for initial processing.
• Use parallel chains for whenever the order or execution is unimportant.
• End with sequential chains for summarization and getting a final text / object output.

4. Documentation Add field descriptions using %. This is not only for readability, but also for the LLM to understand the context of the field. It is basically a part of the prompting process.

out:
  review: str % A comprehensive analysis of the text

Advanced Features

Masks

For parallel-to-sequential transitions, use masks to format data:

mask:
  type: auto
  variables:
    - result.field1
    - result.field2

A template is automatically generated based on the supplied variables to the mask. This template is used to format the data before passing it to the final chainlink.

Caching

The system automatically caches generated prompts, and masks. Improving performance for repeated runs of the same chains.

Configuring the ChainFactory Engine

ChainFactory Engine can be configured using the ChainFactoryEngineConfig class. You can control aspects such as the language model used, caching behavior, concurrency, and execution traces. Below are the configuration options available:

• model: Specifies the model to use (default is "gpt-4o").
• temperature: Sets the temperature for the model, which controls the randomness of the outputs (default is 0).
• cache: Enables caching of prompts and results (default is False).
• provider: Defines the provider for the language model, with supported options including "openai", "anthropic", and "ollama".
• max_tokens: Specifies the maximum tokens allowed per response (default is 1024).
• model_kwargs: A dictionary of additional keyword arguments to pass to the model.
• max_parallel_chains: Sets the maximum number of chains that can execute in parallel (default is 10).
• print_trace: If True, enables printing of execution traces (default is False).
• print_trace_for_single_chain: Similar to print_trace but for single chain execution (default is False).
• pause_between_executions: If True, prompts for confirmation before executing the next chain (default is True).

To use these configurations, you can pass them when initializing the ChainFactoryEngine:

from chainfactory.core.engine.chainfactory_engine import ChainFactoryEngine, ChainFactoryEngineConfig

config = ChainFactoryEngineConfig(
    model="gpt-4o",
    temperature=0.7,
    cache=True,
    provider="openai",
    max_tokens=2048,
    max_parallel_chains=5,
    print_trace=True,
    prompt_between_executions=True
)

engine = ChainFactoryEngine.from_file("examples/haiku.fctr", config)

This configuration allows you to fine-tune the behavior of the ChainFactory to suit your needs.

Conclusion

ChainFactory makes it easy to create complex LLM workflows without writing code. Its simple syntax, automatic prompt generation, and smart features let you focus on what matters - designing great AI workflows.

Remember that this is just an overview - experiment with the examples to discover more possibilities!

Getting Help

• Check the examples folder for more patterns
• Use the built-in validation to catch errors early
• Break complex chains into smaller, reusable pieces
• Ping me directly on email: [e-mail]