jentic-apitools-jobs 1.0.0a16

Jentic Apitools Jobs - Async job execution

Jentic API Tools - Jobs

Async job execution with pluggable backends for running pipeline operations in the background. Supports in-memory thread pool (default) and optional Redis/Celery distributed execution.

Key Features

The jobs package provides a TaskRunner abstraction with two implementations: ThreadPoolRunner for in-memory execution using a thread pool, and CeleryRunner for distributed execution via Redis and Celery. A create_task_runner factory selects the appropriate runner based on configuration. Job state is tracked through JobStorage (with MemoryJobStorage and Redis-backed implementations). Progress is reported through ProgressReporter objects that support SSE streaming to API clients. Periodic cleanup removes expired jobs and optionally their output directories.

Dependencies

Internal: jentic.apitools.common, jentic.apitools.pipelines. Optional: redis, celery[redis].

Installation

pip install jentic-apitools-jobs

Quick Start

from jentic.apitools.jobs import create_task_runner
from jentic.apitools.common.models import JobOperation

runner = create_task_runner()
runner.storage.create_job("job-1", JobOperation.SCORE)
runner.submit_job("job-1", JobOperation.SCORE, {"spec_url": "https://example.com/api.json"})

Testing

pytest tests -v