tokens-lab
LLM utilities with agents, file parsing, and preprocessing built in.
Features
- Agent Framework: LangGraph-based agents with OpenAI integration
- File Text I/O: Parse PDFs, DOCX, PPTX, and images with ease
- Preprocessing: Text normalization and cleaning utilities
- Azure Integration: Azure Storage and service utilities
- ML Tools: String similarity and text analysis
- Utilities: Date, Excel, JSON, and Pandas helpers
- Logging: Structured logging support
- Exception Handling: Custom exception hierarchy
Installation
pip install tokens-lab
Quick Start
LLM Agent
from llm_lab.agent import Agent
from llm_lab.agent.litellm_client import LiteLLMClient
client = LiteLLMClient(model="gpt-4", api_key="your-api-key")
agent = Agent(client=client)
response = agent.process("Hello, how can you help me?")
File Parsing
from llm_lab.filetextio import parsers
content = parsers.parse_document("document.pdf")
Text Preprocessing
from llm_lab.preprocessing import text_normalization
clean_text = text_normalization.normalize_text("messy text ")
Documentation
For complete documentation, visit: https://pwc-me-adv-strategyand.github.io/LLM_Lab/
Requirements
- Python >= 3.10
License
MIT
2 maintainers
Release files for tokens-lab 0.6.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| tokens_lab-0.6.0.tar.gz | 22.6 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| tokens_lab-0.6.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 52.6 kB
Release files / tokens_lab-0.6.0.tar.gz
| Download URL | tokens_lab-0.6.0.tar.gz |
|---|---|
| Size | 22.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
d20ef07aeb126f5920f5f08211fe767b848a8929f1eedbb52018cb78b88f1a00
|
|
BLAKE2b-256 checksum How to use checksums |
64c2ddb2479332af7f472d25a796d695634d8f91edac6c57cc5c267d2ca3a704
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.10.6
|
Release files / tokens_lab-0.6.0-py3-none-any.whl
| Download URL | tokens_lab-0.6.0-py3-none-any.whl |
|---|---|
| Size | 30.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
ab3f4dd96f3da4ced6369b4fdcd646b9ac2b5c41f7a1933128ae7a0b37b9a4b9
|
|
BLAKE2b-256 checksum How to use checksums |
bb96508b06187d2f8a484edcb411760cc985ee1d3048f3b9dfb325b49b66eace
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.10.6
|
