pyrtex 0.1.4

A Python library for batch text extraction and processing using Google Cloud Vertex AI

PyRTex

A simple Python library for batch text extraction and processing using Google Cloud Vertex AI.

PyRTex makes it easy to process multiple documents, images, or text snippets with Gemini models and get back structured, type-safe results using Pydantic models.

✨ Features

• 🚀 Simple API: Just 3 steps - configure, submit, get results
• 📦 Batch Processing: Process multiple inputs efficiently
• 🔒 Type Safety: Pydantic models for structured output
• 🎨 Flexible Templates: Jinja2 templates for prompt engineering
• ☁️ GCP Integration: Seamless Vertex AI and BigQuery integration
• 🧪 Testing Mode: Simulate without GCP costs

📦 Installation

Install from PyPI (recommended):

pip install pyrtex

Or install from source:

git clone https://github.com/CaptainTrojan/pyrtex.git
cd pyrtex
pip install -e .

For development:

pip install -e .[dev]

🚀 Quick Start

from pydantic import BaseModel
from pyrtex import Job

# Define your data structures
class TextInput(BaseModel):
    content: str

class Analysis(BaseModel):
    summary: str
    sentiment: str
    key_points: list[str]

# Create a job
job = Job(
    model="gemini-2.0-flash-lite-001",
    output_schema=Analysis,
    prompt_template="Analyze this text: {{ content }}",
    simulation_mode=True  # Set to False for real processing
)

# Add your data
job.add_request("doc1", TextInput(content="Your text here"))
job.add_request("doc2", TextInput(content="Another document"))

# Process and get results
for result in job.submit().wait().results():
    if result.was_successful:
        print(f"Summary: {result.output.summary}")
        print(f"Sentiment: {result.output.sentiment}")
    else:
        print(f"Error: {result.error}")

📋 Core Workflow

PyRTex uses a simple 3-step workflow:

1. Configure & Add Data

job = Job[YourSchema](model="gemini-2.0-flash-lite-001", ...)
job.add_request("key1", YourModel(data="value1"))
job.add_request("key2", YourModel(data="value2"))

2. Submit & Wait

job.submit().wait()  # Can be chained

3. Get Results

for result in job.results():
    if result.was_successful:
        # Use result.output (typed!)
    else:
        # Handle result.error

⚙️ Configuration

For Simulation Mode (No GCP Required)

job = Job(
    model="gemini-2.0-flash-lite-001",
    output_schema=YourSchema,
    prompt_template="Your prompt",
    simulation_mode=True  # No GCP setup needed
)

For Production (GCP Required)

Set your GCP project ID:

export GOOGLE_PROJECT_ID="your-project-id"

Or configure directly in code:

from pyrtex.config import InfrastructureConfig

config = InfrastructureConfig(project_id="your-project-id")
job = Job(
    model="gemini-2.0-flash-lite-001",
    output_schema=YourSchema,
    prompt_template="Your prompt",
    config=config,
    simulation_mode=False
)

Then authenticate with GCP:

gcloud auth application-default login

Troubleshooting

Error: "Project was not passed and could not be determined from the environment"

This happens when GCP project ID is not set. You have three options:

1. Use simulation mode (recommended for testing):

   simulation_mode=True  # No GCP setup needed
   

2. Set environment variable:

   export GOOGLE_PROJECT_ID="your-project-id"
   

3. Configure in code:

   from pyrtex.config import InfrastructureConfig
   config = InfrastructureConfig(project_id="your-project-id")
   job = Job(..., config=config)
   

📚 Examples

The examples/ directory contains complete working examples:

cd examples

# Generate sample files
python generate_sample_data.py

# Extract contact info from business cards
python 01_simple_text_extraction.py

# Parse product catalogs  
python 02_pdf_product_parsing.py

# Extract invoice data from PDFs
python 03_image_description.py

Example Use Cases

• 📇 Business Cards: Extract contact information
• 📄 Documents: Process PDFs, images (PNG, JPEG)
• 🛍️ Product Catalogs: Parse pricing and inventory
• 🧾 Invoices: Extract structured financial data
• 📊 Batch Processing: Handle multiple files efficiently

🧪 Development

Running Tests

# All tests (mocked, safe)
./test_runner.sh

# Specific test types
./test_runner.sh --unit
./test_runner.sh --integration
./test_runner.sh --flake

# Real GCP tests (costs money!)
./test_runner.sh --real --project-id your-project-id

Windows users:

test_runner.bat --unit
test_runner.bat --flake

Code Quality

• flake8: Linting
• black: Code formatting
• isort: Import sorting
• pytest: Testing with coverage

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Run tests: ./test_runner.sh
5. Submit a pull request

📄 License

MIT License - see LICENSE for details.

🆘 Support

• Issues: GitHub Issues
• Examples: Check the examples/ directory
• Testing: Use simulation_mode=True for development

Common Issues

"Project was not passed and could not be determined from the environment"

• Solution: Set GOOGLE_PROJECT_ID environment variable or use simulation_mode=True

"Failed to initialize GCP clients"

• Solution: Run gcloud auth application-default login or use simulation mode

Examples not working

• Solution: Run python generate_sample_data.py first to create sample files