LLMdantic is a Python package that provides structured interaction with LLMs.
Project description
Structured Output Is All You Need!
LLMdantic redefines the integration of Large Language Models (LLMs) into your projects, offering a seamless, efficient, and powerful way to work with the latest advancements in AI. By abstracting the complexities of LLMs, llmdantic allows developers to focus on what truly matters: building innovative applications.
- Ease of Use. Simplify your LLM interactions. Forget about the intricacies of prompts and models; define your requirements and let llmdantic handle the rest.
- Data Integrity. With Pydantic, define input and output models that ensure your data aligns perfectly with your requirements, maintaining structure and validation at every step.
- Modular and Extensible. Easily switch between different LLMs and customize your experience with the modular and extensible framework provided by llmdantic.
- Cost Tracking. Keep track of your LLM usage and costs, ensuring you stay within budget and optimize your usage.
- Batch Processing. Process multiple data points in a single call, streamlining your operations and enhancing efficiency.
- Retry Mechanism. Automatically retry failed requests, ensuring you get the results you need without any hassle.
Getting Started 🌟
Installation
pip install llmdantic
Usage
- Define Your Models
- inp_model: Define the structure of the data you want to process.
- out_model: Define the structure of the data you expect to receive.
- Use Pydantic to define your models and add custom validation rules.
- Custom validation rules are used to ensure the integrity and quality of your data.
- Add docstrings to your custom validation rules to provide prompts for the LLM.
from pydantic import BaseModel, field_validator
class SummarizeInput(BaseModel):
text: str
class SummarizeOutput(BaseModel):
summary: str
@field_validator("summary")
def summary_must_not_be_empty(cls, v) -> bool:
"""Summary cannot be empty""" # Add docstring that explains the validation rule. This will be used as a prompt for the LLM
if not v.strip():
raise
return v
@field_validator("summary")
def summary_must_be_short(cls, v) -> bool:
"""Summary must be less than 100 words""" # Add docstring that explains the validation rule. This will be used as a prompt for the LLM
if len(v.split()) > 100:
raise
return v
- Initialize LLMdantic
- Initialize LLMdantic with your input and output models.
- Also, provide a objective for the LLM to understand the task.
from llmdantic import LLMdantic, LLMdanticConfig
from langchain_openai import OpenAI
from langchain.llms.base import BaseLanguageModel
llm: BaseLanguageModel = OpenAI()
config: LLMdanticConfig = LLMdanticConfig(
objective="Summarize the text",
inp_model=SummarizeInput,
out_model=SummarizeOutput,
retries=3
)
llmdantic = LLMdantic(
llm=llm,
config=config
)
- Process Your Data
-
Use the
invoke
orbatch
method to process your data. -
invoke
returns an instance ofLLMdanticResult
which contains:text
: The output of the LLM.output
: The output model with the processed data.retry_count
: The number of retries made to get the result.cost
: The cost of the request.inp_tokens
: The number of tokens used for the input.out_tokens
: The number of tokens used for the output.successful_requests
: The number of successful requests made.
-
batch
returns a list ofLLMdanticResult
for each input data.
from llmdantic import LLMdanticResult
data: SummarizeInput = SummarizeInput(text="A long article about natural language processing...")
result: LLMdanticResult = llmdantic.invoke(data)
if result.output:
print(result.output.summary)
- For batch processing, pass a list of input data.
data: List[SummarizeInput] = [
SummarizeInput(text="A long article about natural language processing..."),
SummarizeInput(text="A long article about computer vision...")
]
results: List[Optional[SummarizeOutput]] = llmdantic.batch(data)
for result in results:
if result:
print(result.summary)
- Track Costs and Stats:
- Use the
cost
attribute of theLLMdanticResult
to track the cost of the request. - Use the
usage
attribute of theLLMdantic
to track the usage stats overall.
from llmdantic import LLMdanticResult
data: SummarizeInput = SummarizeInput(text="A long article about natural language processing...")
result: LLMdanticResult = llmdantic.invoke(data)
if result.output:
print(result.output.summary)
# Track the cost of the request
print(f"Cost: {result.cost}")
# Track the usage stats
print(f"Usage: {llmdantic.usage}")
Advanced Usage 🛠
LLMdantic is built on top of the langchain
package, which provides a modular and extensible framework for working
with LLMs. You can easily switch between different LLMs and customize your experience.
Switching LLMs:
- LLMdantic uses the
OpenAI
LLM by default. You can switch to a different LLM by providing an instance of the desired LLM.
from llmdantic import LLMdantic, LLMdanticConfig
from langchain_community.llm.ollama import Ollama
from langchain.llms.base import BaseLanguageModel
llm: BaseLanguageModel = Ollama()
config: LLMdanticConfig = LLMdanticConfig(
objective="Summarize the text",
inp_model=SummarizeInput,
out_model=SummarizeOutput,
retries=3
)
llmdantic = LLMdantic(
llm=llm,
config=config
)
Contributing 🤝
Contributions are welcome! Whether you're fixing bugs, adding new features, or improving documentation, your help makes LLMdantic better for everyone. Feel free to open an issue or submit a pull request.
License 📄
LLMdantic is released under the MIT License. Feel free to use it, contribute, and spread the word!
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for llmdantic-1.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b91ad35fa2d06198e09395e92f7a3c72b7f64ad608d9af0c3e89b53a3c3a801e |
|
MD5 | 31abedbd9544e8490a155122298882db |
|
BLAKE2b-256 | 3da9903be391f4b09c664e0f9b07d8370ba61ff7271b49d4c8dcfc6d76157507 |