llyra 0.2.0

A lightweight interface for calling LLMs with strategy.

Llyra

Lightweight LLaMA Reasoning Assistant

---

✨ Features

• Hybrid Backend Support
  Use local llama-cpp-python or connect to a remote Ollama endpoint via the same interface.

• Minimal, Configurable Inference
  Load prompts, model parameters, and tools from external files.

• Prompt Engineering Friendly
  Easily manage system prompts, roles, and chat formats through external .txt files.

• Optional RAG Integration (Coming Soon)
  Native support for Weaviate-based retrieval-augmented generation.

• Tool Support (Planned)
  Enable LLMs to use JSON-defined tools (function-calling style) with one argument.

---

⚙️ Dependencies

Llyra does not bundle any backend inference engines. You must install them manually according to your needs:

Required (choose one):

• For local models: llama-cpp-python
• For remote inference: any Ollama-compatible API

📦 Installation

pip install llyra

---

🚀 Quickstart

1. Make directary configs/ in your project root.
2. Add config.toml and stategy.toml to configs/ directory.
3. Make your first iterative chat inference with follwing example:

from llyra import Llyra

model = Llyra(mode='local')

response = model.chat('Evening!',keep=True)

print(response)

---

🧩 Function APIs

Initialize Instance

A simple unified interface provides through the Llyra class.

• mode argument is necessary to decide inference locally or remotely when initialize instance.

  It's the only chance you can choose the backend, bachend can't be changed in runtime.

• path argument is optional to override default path to config file when initialize instance.

  The default path is ./configs/config.toml. Be noticed that toml is the only valid config file format.

Here provide simple demo showing how to initialize Llyra instance:

Inference Locally

from llyra import Llyra

model = Llyra(mode='local')

Inference Remotely

from llyra import Llyra

model = Llyra(mode='remote')

Execute Inference

Llyra provides two method to execute single call inference and iterative chat inference.

call() method

call() method provides a simple interface to execute single call inference.

• input argument will take a string as the prompt content for model inference.

It will return a string as the response of model inference.

Here provide a simple demo showing how to execute single call inference:

response = model.call('Evening!')

print(response)

chat() method

chat() method provides a simple interface to execute iterative chat inference.

• message argument will take a string as the current input content for model inference.
• keep argument will take a boolean as the choice of whether keeping current section's content.
  • Set keep to True to keep the current section's content.
  • Set keep to False to start a new section from this call.

  Yes, you don't need to handle the content, Llyra can do that.

It will return a string as the response of the inference's model reply.

Here provide a simple demo showing how to execute iterative chat inference:

response = model.chat('Evening!',True)

print(response)

Get log

Llyra record inference log internally with a custome format which isn't read-friendly for user.

To get readable log record, please using get_log() method.

get_log() method provides a simple interface to extract log record and convert it into readable format without affecting internal log records.

• id argument will take a integer as the index of log record.

  • Set id to a positive value to get specific log record.

    It will raise IndexError when id value out of range.

  • Set id to a negative value to get all log records.

  It will return a dictionary when getting a specific log record, and a list of dictionaries when getting all log records.

Llyra starts its log's id from 0.

Here provide a simple demo showing how to get a specific log record in readable format:

log = model.get_log(1)

print(log)

And, the individual log record should be looked like as:

{
  'id': 1,
  'type': 'call',
  'model': 'llama-2',
  'addition': 'You are a kind assistant.',
  'role': {
    'prompt': 'system',
    'input': 'user',
    'output': 'assistant'
    },
  'iteration': [
    {'query': 'Evening!','response': 'Evening, how can I help you today?'}
    ],
  'temperature': 0.6,
  'create_at': 1750742992.32208
  }

---

🛠 Configuration Example

config.toml

[global]
strategy = "configs/strategy.toml"

[local]
format = "llama-2"
gpu = true
ram = false

[local.model]
name = "Distill-Llama-8B"
directory = "models/"
suffix = ".gguf"


[remote]
model = "llama-2"

[remote.server]
url = "http://localhost"
port = 11434
endpoint = "api/"

strategy.toml

[call]
stop = "<EOF>"
temperature = 0.6

[chat]
prompt = "prompts/prompt.txt"
stop = "<EOF>"
temperature = 0.6

[chat.role]
prompt = "system"
input = "user"
output = "assistant"

---

🧭 Roadmap

Phase	Feature	Status
1	Minimal llama-cpp-python local chat	✅ Finished
2	Predefined prompts via .txt / .json	✅ Finished
3	Ollama remote API support	✅ Finished
4	Section & Branch control.	⏳ Planned
5	Weaviate RAG support	⏳ Planned
6	Tool/function-calling via JSON	⏳ Planned

---

🪪 License

This project is licensed under the MIT License.

---

📚 Attribution

Currently, this package is built on top of the following open-source libraries:

• llama-cpp-python — licensed under the MIT License
  Python bindings for llama.cpp

This package does not include or redistribute any third-party source code.
All dependencies are installed via standard Python packaging tools (e.g. pip).

We gratefully acknowledge the authors and maintainers of these libraries for their excellent work.

---

🌐 About the Name

Llyra is inspired by the constellation Lyra, often associated with harmony and simplicity.
In the same way, this package aims to bring harmony between developers and language models.

---

Designed with care. Built for clarity.