llmfy 0.10.0

LLMfy is a flexible and developer-friendly framework designed to streamline the creation of applications powered by large language models (LLMs). It provides essential tools and abstractions that simplify the integration, orchestration, and management of LLMs across various use cases, enabling developers to focus on building intelligent, context-aware solutions without getting bogged down in low-level model handling. With support for modular components, prompt engineering, and extensibility, LLMfy accelerates the development of AI-driven applications from prototyping to production.

See complete documentation at https://llmfy.readthedocs.io/

How to install

• Optional Library:
  • Install anthropic to use Anthropic Claude models (native Messages API) — 🔸 optional.
  • Install openai to use OpenAI models — 🔸 optional.
  • Install boto3 to use AWS Bedrock models — 🔸 optional.
  • Install google-genai to use Google AI (Gemini) models — 🔸 optional.
  • Install numpy to use Embedding, FAISSVectorStore — 🔸 optional.
  • Install faiss-cpu to use FAISSVectorStore — 🔸 optional.
  • Install typing_extensions to use state in FlowEngine — 🔸 optional.
  • Install redis to use RedisCheckpointer — 🔸 optional.
  • Install SQLAlchemy to use SQLCheckpointer — 🔸 optional. SQLCheckpointer supports both sync and async drivers for multiple databases:
    • PostgreSQL (async: asyncpg, sync: psycopg2) — 🔸 optional.
    • MySQL (async: aiomysql, sync: pymysql) — 🔸 optional.
    • SQLite (async: aiosqlite, sync: built-in) — 🔸 optional.
  • Install spacy to use PERSON_NAME/ADDRESS detection in PIIGuard — 🔸 optional. Also requires the xx_ent_pii_sm NER model (not on PyPI — see PII Guard below).

Using UV

uv add llmfy

Using pip

pip install llmfy

Using github

From a specific branch

# main
uv add git+https://github.com/irufano/llmfy.git@main
# or
pip install git+https://github.com/irufano/llmfy.git@main

# dev
uv add git+https://github.com/irufano/llmfy.git@dev
# or
pip install git+https://github.com/irufano/llmfy.git@dev

From a tag

# example tag version 0.4.3
uv add git+https://github.com/irufano/llmfy.git@v0.4.3
# or
pip install git+https://github.com/irufano/llmfy.git@v0.4.3

Github in requirements.txt

git+https://github.com/irufano/llmfy.git@dev

How to use

Model class names follow <Vendor><APIVariant>Model — e.g. OpenAIChatModel vs OpenAIResponsesModel for OpenAI's two APIs, GoogleAIGenerateModel for Google's generate_content API — so the class name always tells you which API it talks to.

Anthropic models

To use AnthropicMessagesModel (native Messages API), requires install "llmfy[anthropic]" and add below config to your env (or pass api_key= to the model instead):

• ANTHROPIC_API_KEY

OpenAI models

To use OpenAIChatModel (Chat Completions) or OpenAIResponsesModel (Responses API), requires install "llmfy[openai]" and add below config to your env (or pass api_key= to the model instead):

• OPENAI_API_KEY

AWS Bedrock models

To use BedrockConverseModel, requires install "llmfy[boto3]" and add below config to your env (or pass aws_access_key_id=, aws_secret_access_key=, aws_bedrock_region= to BedrockConverseModel instead):

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_BEDROCK_REGION

Google AI models

To use GoogleAIGenerateModel, requires install "llmfy[google-genai]" and add below config to your env (or pass api_key= to GoogleAIGenerateModel instead):

• GOOGLE_API_KEY

PII Guard — PERSON_NAME / ADDRESS detection

To use PIIType.PERSON_NAME/PIIType.ADDRESS detection in PIIGuard, install:

# Using UV
uv add "llmfy[spacy]"
uv add https://github.com/irufano/spacy_ner_pii/releases/download/v0.1.0/xx_ent_pii_sm-0.1.0-py3-none-any.whl

# Using pip
pip install "llmfy[spacy]"
pip install https://github.com/irufano/spacy_ner_pii/releases/download/v0.1.0/xx_ent_pii_sm-0.1.0-py3-none-any.whl

xx_ent_pii_sm isn't published to PyPI, so it can't be pulled in as a normal extra — install it manually from the release wheel above.

All other PIITypes work with no extra install. Note that PIIGuard() defaults to detecting every PIIType, including these two (breaking change from earlier versions, where PIIGuard was regex-only) — pass exclude_types=[PIIType.PERSON_NAME, PIIType.ADDRESS] if you don't want this dependency.

Example

LLMfy Example

from llmfy import (
    OpenAIChatModel,
    OpenAIChatConfig,
    LLMfy,
    Message,
    Role,
    LLMfyException,
)

def sample_prompt():
    info = """Irufano adalah seorang software engineer.
    Dia berasal dari Indonesia.
    Kamu bisa mengunjungi websitenya di https:://irufano.github.io"""

    # Configuration
    config = OpenAIChatConfig(temperature=0.7)
    llm = OpenAIChatModel(model="gpt-4o-mini", config=config)

    SYSTEM_PROMPT = """Answer any user questions based solely on the data below:
    <data>
    {info}
    </data>
    
    DO NOT response outside context."""

    # Initialize framework
    framework = LLMfy(llm, system_message=SYSTEM_PROMPT, input_variables=["info"])

    try:
        messages = [Message(role=Role.USER, content="apa ibukota china")]
       
        response = framework.invoke(messages, info=info)
        print(f"\n>> {response.result.content}\n")

    except LLMfyException as e:
        print(f"{e}")


if __name__ == "__main__":
    sample_prompt()

Develop as Contributor

Commit message format

Commit subjects follow Conventional Commits:

<type>: <short summary>

[optional body]

• type is one of: feat, fix, refactor, chore, ci, docs, test.
• For a breaking change, prefix the subject with [breaking-changes], e.g. [breaking-changes] refactor: consolidate thinking config.
• Keep the summary in the imperative mood (e.g. "add", not "added"/"adds").
• The release workflow copies each commit's subject and body verbatim into the GitHub release changelog, so write both to be read standalone (see .github/workflows/release.yml).

Version bump rules (automatic tagging)

Every push to main is scanned by .github/workflows/auto-tag.yml, which tags a new release automatically — no manual git tag needed. The bump is decided per commit subject, in this precedence order (highest across all new commits wins):

A breaking marker always forces MAJOR regardless of type — use it deliberately when a feat or fix must ship as a major version, e.g. [breaking-changes] feat: ... or feat!: ..., both equivalent.

Build package

uv build

Manual / backfill release

Tagging is automatic (see above). To manually cut or re-run a release, use the workflow_dispatch trigger on release.yml (GitHub Actions UI → "Release & Publish" → "Run workflow", tag input vX.Y.Z) instead of pushing a tag by hand.

After deploy on local

After CI creates or moves the tag, your local tag ref may be stale. To sync:

git fetch --tags --force

The --force flag is needed because git fetch --tags alone won't update tags that already exist locally.

Package Development on local

uv sync --group dev --group docs

or

uv sync --all-groups

Check Lints

uvx ruff check --statistics . 2>&1 | tail -60 

Mkdocs run on local

uv sync --group docs

# Serve on local
mkdocs serve

# Build docs
mkdocs build