PythonAI Grounded Generator
A compact Python helper for generating answers from retrieved context with an already loaded Hugging Face causal language model.
It provides:
- Custom system prompts and questions
- Token-budget-aware context truncation
- Qwen3 non-thinking chat formatting
- CPU or CUDA inference (device controlled by the caller)
- A plain-text answer suitable for wrapping in application-specific Pydantic models
Retrieval, context assembly, and structured JSON serialization remain the responsibility of the calling application. This package does not guarantee factual correctness or replace source validation.
Installation
After publication:
pip install pythonai-grounded-generator
Or install the local project:
pip install .
Example
from transformers import AutoModelForCausalLM, AutoTokenizer
from pythonai_grounded import generate_answer
name = "Qwen/Qwen3-0.6B"
tokenizer = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name).to("cpu")
model.eval()
answer = generate_answer(
model=model,
tokenizer=tokenizer,
question="Which endpoint loads an adapter?",
context="Use POST /v1/load_lora_adapter to load an adapter.",
system_prompt="Answer only from the context. Copy technical names exactly.",
max_new_tokens=64,
device="cpu",
)
print(answer)
API
generate_answer(
model,
tokenizer,
question: str,
context: str,
system_prompt: str,
max_new_tokens: int = 64,
device: str = "cpu",
) -> str
The supplied model and tokenizer must be compatible. The model must already be on the requested device and in evaluation mode. Model context limits are read from model.config.max_position_embeddings, falling back to 32768 if absent. Check actual model capabilities when switching models.
Build
uv build
The distributions are created in dist/.
Metadata
Release files for pythonai-grounded-generator 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| pythonai_grounded_generator-0.1.0.tar.gz | 4.4 kB | Without provenance | Details for pythonai_grounded_generator-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| pythonai_grounded_generator-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for pythonai_grounded_generator-0.1.0-py3-none-any.whl |
Total release size: 8.4 kB
Release files / pythonai_grounded_generator-0.1.0.tar.gz
| Download URL | pythonai_grounded_generator-0.1.0.tar.gz |
|---|---|
| Size | 4.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
cf6ebe921faaf5039232cb98070f8da55503e325011409da04416534aeb7b062
|
|
BLAKE2b-256 checksum How to use checksums |
790aaa24800fdbfe4f6de013be27e6627bdd0c87bb0498fa9214acf1074a7b99
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.9.13 {"installer":{"name":"uv","version":"0.9.13"},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"22.04","id":"jammy","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / pythonai_grounded_generator-0.1.0-py3-none-any.whl
| Download URL | pythonai_grounded_generator-0.1.0-py3-none-any.whl |
|---|---|
| Size | 4.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
8c1c509dd900557e6838563001ce519df9bc611031526059e73797af95361b85
|
|
BLAKE2b-256 checksum How to use checksums |
7aa2359673eee62c2c5f64cabe6e0937cd2563a9f4766b29126102713b5e5e0b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.9.13 {"installer":{"name":"uv","version":"0.9.13"},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"22.04","id":"jammy","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Security for pythonai-grounded-generator 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.