sibila 0.2.2

Structured queries from local or online LLM models

Sibila

Extract structured information from LLM models, using a common API to access remote models like GPT-4 or local models via llama.cpp.

• Query structured information into Pydantic BaseModel objects or typed Python dicts.
• Use the same API for local and remote models.
• Thread-based interaction with chat/instruct fine-tuned models.
• Compare output across local/remote models with included utilities, text or CSV output.
• Model management directory: manage models and their configurations and quickly switch between models.
• Automatic chat templates: identifies and uses the right templates for each model.

With Sibila you can extract structured data from a local model like OpenChat-3.5 with 7B params:

from sibila import (LlamaCppModel, OpenAIModel)
from pydantic import BaseModel, Field

class Info(BaseModel):
    event_year: int
    first_name: str
    last_name: str
    age_at_the_time: int
    nationality: str

openchat = LlamaCppModel("models/openchat-3.5-1210.Q5_K_M.gguf")

openchat.query_pydantic(Info,
                        "Just be helpful.", # instructions, aka system message
                        "Who was the first man in the moon?")

Outputs an object of class Info, initialized with the model's output:

Info(event_year=1969,
     first_name='Neil',
     last_name='Armstrong',
     age_at_the_time=38,
     nationality='American')

With the same API you can also query OpenAI models:

gpt4 = OpenAIModel("gpt-4-0613")

gpt4.query_pydantic(Info,
                    "Just be helpful.",
                    "Who was the first man in the moon?")

Which creates an initialized Info object as above.

If Pydantic BaseModel objects are too much for your project, you can also use a very simple convention called dictype, which defines structure and types of output dicts.

Sibila also includes model management and tools to compare output between models.

Examples

The included examples show what you can do with local or remote models in Sibila: structured data extraction, classification, summarization, etc.

Although you can use any llama.cpp or OpenAI supported model, by default most of the examples use OpenChat, a local 7B quantized model which is very capable for information extraction tasks. To use OpenAI models, just uncomment a line or two.

Check the examples here.

Installation and getting started

Sibila can be installed from PyPI by doing:

pip install sibila

For running local models with hardware acceleration, accessing OpenAI and general "getting started" help, see How to get started.

Documentation

The API reference and more info is available here.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Sibila?

Sibila is the Portuguese word for Sibyl. The Sibyls were wise oracular women in ancient Greece. Their mysterious words puzzled people throughout the centuries, providing insight or prophetic predictions.

Michelangelo's Delphic Sibyl, in the Sistine Chapel ceiling.