A utility package for querying language models with pattern matching and retry logic
Project description
llm-loop
Overview
llm-loop
is a Python package designed to simplify the process of querying language models (like GPT or similar models) until a response matching a specified pattern is obtained or a maximum number of attempts is reached. This is particularly useful when working with AI models in scenarios where a specific format of response is required.
Installation
pip install llm-loop
This will install the necessary Python packages, including ctransformers
and any other dependencies.
Usage
Here's a basic example of how to use llm-loop
:
-
Import the necessary modules:
import os from ctransformers import AutoModelForCausalLM, AutoTokenizer from llm_loop.main import LLMLoop
-
Initialize the model with custom parameters:
model_name = "YourModelName" model_file = "YourModelFileName" start_dir = '/path/to/your/model' model_path = f"{start_dir}/{model_file}" llm = AutoModelForCausalLM.from_pretrained(model_name, model_file=model_path, model_type='YourModelType', gpu_layers=YourGPULayers)
-
Create an instance of LLMLoop and query the model:
loop = LLMLoop(llm, 10) # 10 is the maximum number of attempts prompt = "Your prompt here" pattern = r'Your regex pattern here' response = loop.query_llm(prompt=prompt, pattern=pattern) print("Response:", response)
Contributing
Contributions to llm-loop
are welcome! Please feel free to submit pull requests or open issues to suggest improvements or add new features.
License
MIT.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.