llmfy 0.9.3

LLMfy is a flexible and developer-friendly framework designed to streamline the creation of applications powered by large language models (LLMs). It provides essential tools and abstractions that simplify the integration, orchestration, and management of LLMs across various use cases, enabling developers to focus on building intelligent, context-aware solutions without getting bogged down in low-level model handling. With support for modular components, prompt engineering, and extensibility, LLMfy accelerates the development of AI-driven applications from prototyping to production.

See complete documentation at https://llmfy.readthedocs.io/

How to install

• Optional Library:
  • Install anthropic to use Anthropic Claude models (native Messages API) — 🔸 optional.
  • Install openai to use OpenAI models — 🔸 optional.
  • Install boto3 to use AWS Bedrock models — 🔸 optional.
  • Install google-genai to use Google AI (Gemini) models — 🔸 optional.
  • Install numpy to use Embedding, FAISSVectorStore — 🔸 optional.
  • Install faiss-cpu to use FAISSVectorStore — 🔸 optional.
  • Install typing_extensions to use state in FlowEngine — 🔸 optional.
  • Install redis to use RedisCheckpointer — 🔸 optional.
  • Install SQLAlchemy to use SQLCheckpointer — 🔸 optional. SQLCheckpointer supports both sync and async drivers for multiple databases:
    • PostgreSQL (async: asyncpg, sync: psycopg2) — 🔸 optional.
    • MySQL (async: aiomysql, sync: pymysql) — 🔸 optional.
    • SQLite (async: aiosqlite, sync: built-in) — 🔸 optional.
  • Install spacy to use PERSON_NAME/ADDRESS detection in PIIGuard — 🔸 optional. Also requires the xx_ent_pii_sm NER model (not on PyPI — see PII Guard below).

Using UV

uv add llmfy

Using pip

pip install llmfy

Using github

From a specific branch

# main
uv add git+https://github.com/irufano/llmfy.git@main
# or
pip install git+https://github.com/irufano/llmfy.git@main

# dev
uv add git+https://github.com/irufano/llmfy.git@dev
# or
pip install git+https://github.com/irufano/llmfy.git@dev

From a tag

# example tag version 0.4.3
uv add git+https://github.com/irufano/llmfy.git@v0.4.3
# or
pip install git+https://github.com/irufano/llmfy.git@v0.4.3

Github in requirements.txt

git+https://github.com/irufano/llmfy.git@dev

How to use

Model class names follow <Vendor><APIVariant>Model — e.g. OpenAIChatModel vs OpenAIResponsesModel for OpenAI's two APIs, GoogleAIGenerateModel for Google's generate_content API — so the class name always tells you which API it talks to.

Anthropic models

To use AnthropicMessagesModel (native Messages API), requires install "llmfy[anthropic]" and add below config to your env (or pass api_key= to the model instead):

• ANTHROPIC_API_KEY

OpenAI models

To use OpenAIChatModel (Chat Completions) or OpenAIResponsesModel (Responses API), requires install "llmfy[openai]" and add below config to your env (or pass api_key= to the model instead):

• OPENAI_API_KEY

AWS Bedrock models

To use BedrockConverseModel, requires install "llmfy[boto3]" and add below config to your env (or pass aws_access_key_id=, aws_secret_access_key=, aws_bedrock_region= to BedrockConverseModel instead):

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_BEDROCK_REGION

Google AI models

To use GoogleAIGenerateModel, requires install "llmfy[google-genai]" and add below config to your env (or pass api_key= to GoogleAIGenerateModel instead):

• GOOGLE_API_KEY

PII Guard — PERSON_NAME / ADDRESS detection

To use PIIType.PERSON_NAME/PIIType.ADDRESS detection in PIIGuard, install:

# Using UV
uv add "llmfy[spacy]"
uv add https://github.com/irufano/spacy_ner_pii/releases/download/v0.1.0/xx_ent_pii_sm-0.1.0-py3-none-any.whl

# Using pip
pip install "llmfy[spacy]"
pip install https://github.com/irufano/spacy_ner_pii/releases/download/v0.1.0/xx_ent_pii_sm-0.1.0-py3-none-any.whl

xx_ent_pii_sm isn't published to PyPI, so it can't be pulled in as a normal extra — install it manually from the release wheel above.

All other PIITypes work with no extra install. Note that PIIGuard() defaults to detecting every PIIType, including these two (breaking change from earlier versions, where PIIGuard was regex-only) — pass exclude_types=[PIIType.PERSON_NAME, PIIType.ADDRESS] if you don't want this dependency.

Example

LLMfy Example

from llmfy import (
    OpenAIChatModel,
    OpenAIChatConfig,
    LLMfy,
    Message,
    Role,
    LLMfyException,
)

def sample_prompt():
    info = """Irufano adalah seorang software engineer.
    Dia berasal dari Indonesia.
    Kamu bisa mengunjungi websitenya di https:://irufano.github.io"""

    # Configuration
    config = OpenAIChatConfig(temperature=0.7)
    llm = OpenAIChatModel(model="gpt-4o-mini", config=config)

    SYSTEM_PROMPT = """Answer any user questions based solely on the data below:
    <data>
    {info}
    </data>
    
    DO NOT response outside context."""

    # Initialize framework
    framework = LLMfy(llm, system_message=SYSTEM_PROMPT, input_variables=["info"])

    try:
        messages = [Message(role=Role.USER, content="apa ibukota china")]
       
        response = framework.invoke(messages, info=info)
        print(f"\n>> {response.result.content}\n")

    except LLMfyException as e:
        print(f"{e}")


if __name__ == "__main__":
    sample_prompt()

Develop as Contributor

Commit message format

Commit subjects follow Conventional Commits:

<type>: <short summary>

[optional body]

• type is one of: feat, fix, refactor, chore, ci, docs, test.
• For a breaking change, prefix the subject with [breaking-changes], e.g. [breaking-changes] refactor: consolidate thinking config.
• Keep the summary in the imperative mood (e.g. "add", not "added"/"adds").
• The release workflow copies each commit's subject and body verbatim into the GitHub release changelog, so write both to be read standalone (see .github/workflows/release.yml).

Build package

uv build

Trigger build and deploy to PyPI

# TAG_NAME must start with "v" (e.g., v1.0.0)
git tag -a [TAG_NAME] -m "[TAG_MESSAGE]"

# push tag to remote
git push origin [TAG_NAME]

After deploy on local

After the CI moves the tag, your local tag still points to the old commit. To sync:

git fetch --tags --force

The --force flag is needed because git fetch --tags alone won't update tags that already exist locally.

Package Development on local

uv sync --group dev --group docs

or

uv sync --all-groups

Check Lints

uvx ruff check --statistics . 2>&1 | tail -60 

Mkdocs run on local

uv sync --group docs

# Serve on local
mkdocs serve

# Build docs
mkdocs build