Skip to main content

A tool to process large datasets using OpenAI's batch API

Project description

Batch Runner

PyPI License

📌 Overview

Process large datasets efficiently using OpenAI's batch API.


🚀 Installation

Install the latest release via PyPI:

pip install -U hamtaa-batch-runner

⚡ Quick Start

from pydantic import BaseModel
from texttools import BatchRunner, BatchConfig

config = BatchConfig(
    system_prompt="Extract entities from the text",
    job_name="entity_extraction",
    input_data_path="data.json",
    output_data_filename="results.json",
    model="gpt-4o-mini"
)

class Output(BaseModel):
    entities: list[str]

runner = BatchRunner(config, output_model=Output)
runner.run()

🤝 Contributing

Contributions are welcome!
Feel free to open issues, suggest new features, or submit pull requests.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

hamtaa_batch_runner-0.0.1.tar.gz (8.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

hamtaa_batch_runner-0.0.1-py3-none-any.whl (9.1 kB view details)

Uploaded Python 3

File details

Details for the file hamtaa_batch_runner-0.0.1.tar.gz.

File metadata

  • Download URL: hamtaa_batch_runner-0.0.1.tar.gz
  • Upload date:
  • Size: 8.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.8.22

File hashes

Hashes for hamtaa_batch_runner-0.0.1.tar.gz
Algorithm Hash digest
SHA256 528bb29f49633544877893d575ee451489e6a2f6fd208e2994776413cfc49479
MD5 0df4df78bf13699098611f2cfbc3e316
BLAKE2b-256 eddb023b3df76c2d7f19740b1de9b8ac5522f60202c0f122389adfc3095349ea

See more details on using hashes here.

File details

Details for the file hamtaa_batch_runner-0.0.1-py3-none-any.whl.

File metadata

File hashes

Hashes for hamtaa_batch_runner-0.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 692f9f2bb60cff7774cc0d377ffad8e1e21f5a7dabfddeaeb00eeca5eb45b858
MD5 5908cf20b7852ef1cdbe03b3153895db
BLAKE2b-256 10c238ee13b1704057b9df6853c3b6831d232dc753a871983a05c2450acafbd0

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page