llmfy 0.6.3

`LLMfy` is a framework for developing applications with large language models (LLMs).

LLMfy is a flexible and developer-friendly framework designed to streamline the creation of applications powered by large language models (LLMs). It provides essential tools and abstractions that simplify the integration, orchestration, and management of LLMs across various use cases, enabling developers to focus on building intelligent, context-aware solutions without getting bogged down in low-level model handling. With support for modular components, prompt engineering, and extensibility, LLMfy accelerates the development of AI-driven applications from prototyping to production.

See complete documentation at https://llmfy.readthedocs.io/

How to install

• Optional Library:
  • Install openai to use OpenAI models — 🔸 optional.
  • Install boto3 to use AWS Bedrock models — 🔸 optional.
  • Install numpy to use Embedding, FAISSVectorStore — 🔸 optional.
  • Install faiss-cpu to use FAISSVectorStore — 🔸 optional.
  • Install typing_extensions to use state in FlowEngine — 🔸 optional.
  • Install redis to use RedisCheckpointer — 🔸 optional.
  • Install SQLAlchemy to use SQLCheckpointer — 🔸 optional. SQLCheckpointer supports both sync and async drivers for multiple databases:
    • PostgreSQL (async: asyncpg, sync: psycopg2) — 🔸 optional.
    • MySQL (async: aiomysql, sync: pymysql) — 🔸 optional.
    • SQLite (async: aiosqlite, sync: built-in) — 🔸 optional.
  • Install spacy to use PERSON_NAME/ADDRESS detection in PIIGuard — 🔸 optional. Also requires the xx_ent_pii_sm NER model (not on PyPI — see PII Guard below).

Using UV

uv add llmfy

Using pip

pip install llmfy

Using github

From a specific branch

# main
uv add git+https://github.com/irufano/llmfy.git@main
# or
pip install git+https://github.com/irufano/llmfy.git@main

# dev
uv add git+https://github.com/irufano/llmfy.git@dev
# or
pip install git+https://github.com/irufano/llmfy.git@dev

From a tag

# example tag version 0.4.3
uv add git+https://github.com/irufano/llmfy.git@v0.4.3
# or
pip install git+https://github.com/irufano/llmfy.git@v0.4.3

Github in requirements.txt

git+https://github.com/irufano/llmfy.git@dev

How to use

OpenAI models

To use OpenAIModel, requires install "llmfy[openai]" and add below config to your env:

• OPENAI_API_KEY

AWS Bedrock models

To use BedrockModel, requires install "llmfy[boto3]" and add below config to your env:

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_BEDROCK_REGION

Google AI models

To use GoogleAIModel, requires install "llmfy[google-genai]" and add below config to your env:

• GOOGLE_API_KEY

PII Guard — PERSON_NAME / ADDRESS detection

To use PIIType.PERSON_NAME/PIIType.ADDRESS detection in PIIGuard, install:

# Using UV
uv add "llmfy[spacy]"
uv add https://github.com/irufano/spacy_ner_pii/releases/download/v0.1.0/xx_ent_pii_sm-0.1.0-py3-none-any.whl

# Using pip
pip install "llmfy[spacy]"
pip install https://github.com/irufano/spacy_ner_pii/releases/download/v0.1.0/xx_ent_pii_sm-0.1.0-py3-none-any.whl

xx_ent_pii_sm isn't published to PyPI, so it can't be pulled in as a normal extra — install it manually from the release wheel above.

All other PIITypes work with no extra install. Note that PIIGuard() defaults to detecting every PIIType, including these two (breaking change from earlier versions, where PIIGuard was regex-only) — pass exclude_types=[PIIType.PERSON_NAME, PIIType.ADDRESS] if you don't want this dependency.

Example

LLMfy Example

from llmfy import (
    OpenAIModel,
    OpenAIConfig,
    LLMfy,
    Message,
    Role,
    LLMfyException,
)

def sample_prompt():
    info = """Irufano adalah seorang software engineer.
    Dia berasal dari Indonesia.
    Kamu bisa mengunjungi websitenya di https:://irufano.github.io"""

    # Configuration
    config = OpenAIConfig(temperature=0.7)
    llm = OpenAIModel(model="gpt-4o-mini", config=config)

    SYSTEM_PROMPT = """Answer any user questions based solely on the data below:
    <data>
    {info}
    </data>
    
    DO NOT response outside context."""

    # Initialize framework
    framework = LLMfy(llm, system_message=SYSTEM_PROMPT, input_variables=["info"])

    try:
        messages = [Message(role=Role.USER, content="apa ibukota china")]
       
        response = framework.invoke(messages, info=info)
        print(f"\n>> {response.result.content}\n")

    except LLMfyException as e:
        print(f"{e}")


if __name__ == "__main__":
    sample_prompt()

Develop as Contributor

Build package

uv build

Trigger build and deploy to PyPI

# TAG_NAME must start with "v" (e.g., v1.0.0)
git tag -a [TAG_NAME] -m "[TAG_MESSAGE]"

# push tag to remote
git push origin [TAG_NAME]

After deploy on local

After the CI moves the tag, your local tag still points to the old commit. To sync:

git fetch --tags --force

The --force flag is needed because git fetch --tags alone won't update tags that already exist locally.

Package Development on local

uv sync --group dev --group docs

or

uv sync --all-groups

Mkdocs run on local

uv sync --group docs

# Serve on local
mkdocs serve

# Build docs
mkdocs build