Skip to main content

Airflow provider for LLM-powered intelligent task retry

Project description

# airflow-provider-smart-retry

An Apache Airflow provider that uses LLMs to make intelligent retry decisions when tasks fail.

## The Problem

Airflow's built-in retry mechanism is static — it waits the same amount of time and retries blindly regardless of the error type. This leads to:

- Rate limit errors being retried too fast

- Auth errors being retried pointlessly

- Network errors not being retried fast enough

## The Solution

LLMSmartRetryOperator analyzes the error log using a local LLM (via Ollama) and decides:

- **Should we retry at all?** (auth errors → no)

- **How long should we wait?** (rate limits → 60s, network → 0s)

- **What type of error is this?** (rate_limit / network / auth / data_schema / unknown)

## How It Works

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

airflow_provider_smart_retry-0.1.0.tar.gz (3.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

airflow_provider_smart_retry-0.1.0-py3-none-any.whl (3.8 kB view details)

Uploaded Python 3

File details

Details for the file airflow_provider_smart_retry-0.1.0.tar.gz.

File metadata

File hashes

Hashes for airflow_provider_smart_retry-0.1.0.tar.gz
Algorithm Hash digest
SHA256 8cc8b41d802acc9289fa1e73152297dde5b03ccc2e7ec2296aad6f4060e75da8
MD5 95ea61a0ec7d2fa8c2c4d70f7e274646
BLAKE2b-256 d326d4d62da751f9081b22224408e3273d02935b2ae32d0a1bdbe1a8bdb55ec8

See more details on using hashes here.

File details

Details for the file airflow_provider_smart_retry-0.1.0-py3-none-any.whl.

File metadata

File hashes

Hashes for airflow_provider_smart_retry-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 f87956c382b3f6d43743756715824b0f5798c2626a1da25a17f6f34575a57f54
MD5 de0debc97df93207ba84c216a5d44c05
BLAKE2b-256 fc37a208413d4c26ccd73b46d93c2deeee6abf95c69dc87404e4d9a69be24c26

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page