pleonasty 0.3.5

A very simple abstraction for LLMs to get single responses to a given input.

Pleonast

Pleonast is a Python library built on top of vLLM and Hugging Face Transformers that streamlines batch annotation, analysis, and interactive chat with large language models. It makes it easy to apply the same prompt or system instructions to hundreds or thousands of texts and save results in CSV form, or to start an interactive chat loop for ad-hoc experimentation.

Key Features

• Batch Annotation: Annotate large text datasets (CSV or Python lists) with a custom LLM prompt.
• CSV I/O: Read input columns, retain metadata, and write annotated outputs as CSV.
• Chat Mode: Launch an interactive REPL for back‑and‑forth conversation with your model.
• Token‑based Chunking: Split long documents into N‑token chunks to avoid overflow.
• Flexible Model Loading: Support for in‑flight 4‑bit quantization via BitsAndBytes, tensor‑parallel across multiple GPUs, CPU offload, and Hugging Face Hub authentication.

Installation

pip install pleonast

GPU & vLLM Requirements

• Python 3.10+
• PyTorch with CUDA support (if using GPU inference)
• vLLM v0.9.1+

Set environment variables before importing pleonast or vllm if you need custom cache locations:

# Hugging Face cache (models, tokenizers):
export HF_HOME=/data/models/hf
# vLLM internal cache (kernels, metadata):
export VLLM_CACHE_ROOT=/data/models/vllm_cache

Quickstart

1. Initialize Pleonast

from pleonasty import Pleonast

# Load a 7B instruct model in 4-bit, auto‑sharded on all GPUs
ple = Pleonast(
    model="meta-llama/Llama-3.1-7B-Instruct",
    quantize_model=True,
    tensor_parallel_size=4,
    gpu_memory_utilization=0.9,
    trust_remote_code=True,
    hf_token="<YOUR_HF_TOKEN>"  # for private repos
)

2. Batch Analyze Python List

texts = ["Hello world!", "The capital of France is"]
sampling = {"temperature": 0.01, "max_tokens": 20, "top_p": 0.95}
ple.batch_analyze_to_csv(
    texts=texts,
    text_metadata={"id": [1, 2]},
    output_csv="out.csv",
    chunk_into_n_tokens=1024,
    **sampling
)
# writes a CSV with columns: id, text, Input_WC, LLM_Response

3. Batch Analyze a CSV File

ple.batch_analyze_csv_to_csv(
    input_csv="data/input.csv",
    text_columns_to_process=["post_text"],
    metadata_columns_to_retain=["user_id", "timestamp"],
    output_csv="data/annotated.csv",
    chunk_into_n_tokens=2048,
    temperature=0.01,
    max_tokens=512,
    top_k=10
)

4. Interactive Chat Mode

# Launch a REPL chat
ple.chat_mode(
    temperature=0.7,
    top_k=50,
    max_tokens=100,
    bot_name="Annotator",
    system_prompt="You are an expert psychological annotator."
)
# Type your messages, type 'quit' to exit.

5. Other Examples

I have included an example notebook is included in this repo that shows how it can be used. I have also included a "chat mode" where you can load up an LLM and have back-and-forth interactions with it — an example of this is also provided in a sample notebook.

Advanced Usage

Setting Message Context Programmatically

# Few-shot or system prompts for all calls
msgs = [
    {"role": "system", "content": "Annotate for sentiment."},
    {"role": "user",   "content": "Example: I love this!"},
    {"role": "assistant", "content": "POSITIVE"}
]
ple.set_message_context(msgs)

Loading Context from CSV

ple.set_message_context_from_CSV("prompts/context.csv")

Token‑level Chunking

chunks = ple.chunk_by_tokens(long_text, chunk_size=2000)

Converting Prompt to Template String

template_str = ple.convert_prompt_to_template_str(msgs)

API Reference

Method	Description
Pleonast(...)	Initialize engine with model, quantization, etc.
ple.set_message_context(msgs)	Set chat context as list of role/content dicts
ple.set_message_context_from_CSV(file)	Load context prompts from CSV (role,content)
ple.chunk_by_tokens(text, n)	Split text into n-token chunks
ple.analyze_text(texts, **params)	Annotate list of texts, returns LLM_Result objects
ple.batch_analyze_to_csv(...)	Annotate Python list & write CSV
ple.batch_analyze_csv_to_csv(...)	Annotate CSV file & write CSV
ple.chat_mode(...)	Launch interactive chat REPL
ple.convert_prompt_to_template_str	Serialize messages via HF tokenizer’s chat template

Contributing

Contributions, bug reports, and feature requests are welcome! Please open issues or pull requests at https://github.com/ryanboyd/pleonasty

License

MIT License