Duckle

Local-first ETL/ELT pipelines that run on DuckDB. Python builds the plan, DuckDB moves the rows. Your data never leaves the machine.

PyPI License GitHub

Build your first pipeline

Paste this into Claude Code, Cursor, or Codex. The agent does the rest.

Run uvx duckle quickstart to build my first pipeline and run it

Nothing to install first. The agent fetches Duckle and the DuckDB engine on demand, runs a real pipeline, and shows you the rows.

Prefer to drive it yourself:

uvx duckle quickstart      # no install
pip install duckle         # or install it
Terminal: uvx duckle quickstart scaffolds sample data and a pipeline, runs it, and prints the resulting rows

quickstart scaffolds sample data and a pipeline, runs it, and shows you the rows. One command from nothing to a result, because the engine comes with it: a ~20 MB native binary plus the DuckDB CLI. No JVM, no Docker, no server, no account.

Or install it

Terminal: pip install duckle brings the DuckDB engine, then a job.py using the Python API reads a CSV, filters, derives a column and writes Parquet

Where uvx is for trying it, pip is for keeping it: a persistent duckle command, duckle-mcp for agents, and import duckle in Python.

No data passes through Python

Every method appends a node to a pipeline graph. Nothing executes until .run(), and then the whole graph is handed to the engine, compiled to SQL, and executed inside DuckDB.

That is the difference from a dataframe library: a billion-row job costs what the SQL costs, because it is SQL. There is no interpreter in the data path, and no to_pandas() escape hatch quietly pulling rows into memory.

import duckle
from duckle import col

(duckle.read_csv("orders.csv")
    .where(col.amount >= 20)
    .derive(total="round(amount * 1.2, 2)", tag="f'{region}-{id}'")
    .write_parquet("out.parquet")
    .run())

Python expressions, compiled to SQL

amount * 1.2, f'{a}-{b}', x if c else y, name.strip().upper() and email is None are translated to vectorized DuckDB SQL at plan time:

.derive(band="'high' if amount > 50 else 'low'")
# CASE WHEN ("amount" > 50) THEN 'high' ELSE 'low' END

An expression with no exact SQL equivalent is rejected with the construct named, never quietly executed somewhere slower. Comprehensions, lambda, indexing and eval are refused by name.

Prefer editor completion over strings? col builds the same tree with real operators, and fragments are reusable:

eu_only = col.region.isin(["EU", "UK"])
p.where(eu_only & (col.amount >= 20))

See exactly what will run, before it runs:

p.explain()   # prints the compiled SQL, one block per stage

360+ components, not 10 file formats

Component ids map onto attribute paths, so anything in the catalog is reachable:

duckle.src.salesforce(object="Account", authMode="clientCredentials")
duckle.xf.geo.reproject(geomColumn="geom", targetCrs="EPSG:3857")
duckle.snk.salesforce.bulk(operation="upsert", externalIdField="Ext__c")

104 sources, 66 sinks and 138 transforms: Postgres, MySQL, Oracle, SQL Server, Snowflake, Databricks, Teradata, SAP OData, Salesforce (including Bulk API 2.0), Kafka, WebSocket, S3, SFTP, IMAP, LanceDB and more.

duckle.component_ids(contains="salesforce")
duckle.describe("snk.salesforce.bulk")   # settings, and which ones the engine ignores

A CI gate that needs no engine

Terminal: duckle validate reports one failing and one passing pipeline, then exits 1

duckle validate compiles every pipeline without opening a source or writing a sink, so it needs no DuckDB, no credentials and no network. Exit codes are stable and safe to gate on:

code meaning
0 clean
1 a real finding: a pipeline failed, or did not compile
2 the runner could not start: bad usage, missing engine 
duckle validate --json          # machine-readable, for a build step
duckle --pipeline my.json       # run one

Note: validate does not yet catch every missing required property, so a clean validate is not proof that a run will succeed.

Agent-ready, with nothing installed

Terminal: adding duckle as an MCP server via uvx, then an agent discovering Salesforce connectors, reading a component schema, creating a pipeline that compiles, and running it

The same package is an MCP server, so an AI agent gets a governed way to work with data instead of a shell. Point Claude Code, Claude Desktop or Cursor at it:

{ "mcpServers": { "duckle": { "command": "uvx", "args": ["duckle", "mcp"] } } }

Or in Claude Code:

claude mcp add duckle -- uvx duckle mcp

That is the whole setup. No pip install, no PATH, no engine to configure: uv fetches the package and the DuckDB engine into a throwaway environment and the server finds it there. If you did pip install duckle, "command": "duckle", "args": ["mcp"] works the same way.

19 tools, including list_components, get_component_schema, create_pipeline, validate_pipeline, run_pipeline, pipeline_lineage, trust_report and schema_drift.

What that buys over letting an agent write a script: it can discover a real connector rather than guess one, compile-check a pipeline before anything executes, run it, and get column-level lineage back. validate_pipeline opens no source and writes no sink, so an agent can check its work without touching your data. The pipeline it produces is the same JSON your desktop canvas opens, so you can inspect what it built.

Code and canvas are the same file

Pipelines are the same JSON the Duckle desktop studio reads. A pipeline written in Python opens on the canvas, and one drawn on the canvas runs from Python:

p.save("pipelines/orders.json")          # opens in the studio
duckle.from_json("pipelines/orders.json").run()

Install notes

duckle depends on duckdb-cli, published by the DuckDB Foundation, so the engine arrives with the install and works offline. To pin your own build instead, set DUCKLE_DUCKDB_BIN.

Wheels ship for Linux, macOS and Windows on x86-64 and arm64. The wheel carries a compiled Rust binary and is Python-version independent (py3-none-<platform>).

Apache-2.0  ·  GitHub  ·  duckle.org  ·  Issues

Project links

Data verified by PyPI on Aug 30, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for souravetl from gravatar.com souravetl

Credits

Author: Slothflow Labs

GitHub Statistics

Data verified by PyPI on Aug 30, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

Apache Software License (Apache-2.0)

Requires

Python >=3.8

Tags

etl elt duckdb data-engineering pipeline data-integration

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

duckle-0.7.1-py3-none-win_arm64.whl (26.3 MB view details)

Uploaded Python 3Windows ARM64

duckle-0.7.1-py3-none-win_amd64.whl (27.6 MB view details)

Uploaded Python 3Windows x86-64

duckle-0.7.1-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (27.7 MB view details)

Uploaded Python 3manylinux: glibc 2.17+ x86-64

duckle-0.7.1-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl (26.6 MB view details)

Uploaded Python 3manylinux: glibc 2.17+ ARM64

duckle-0.7.1-py3-none-macosx_11_0_x86_64.whl (25.6 MB view details)

Uploaded Python 3macOS 11.0+ x86-64

duckle-0.7.1-py3-none-macosx_11_0_arm64.whl (24.5 MB view details)

Uploaded Python 3macOS 11.0+ ARM64

File details

Details for the file duckle-0.7.1-py3-none-win_arm64.whl.

File metadata

  • Download URL: duckle-0.7.1-py3-none-win_arm64.whl
  • Upload date:
  • Size: 26.3 MB
  • Tags: Python 3, Windows ARM64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for duckle-0.7.1-py3-none-win_arm64.whl
Algorithm Hash digest
SHA256 b1922603509b8235825d91ea3e9f0653586955a10ae03c9a1b1c8d99097ca237
MD5 25e2243662e0345400209e79e0ec7e9f
BLAKE2b-256 c49e8ff2745eed64b89113629a9911d974ac9b7bf55681badabeab64fe679c4f

See more details on using hashes here.

Provenance

The following attestation bundles were made for duckle-0.7.1-py3-none-win_arm64.whl:

Publisher: pypi.yml on slothflowlabs/duckle

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file duckle-0.7.1-py3-none-win_amd64.whl.

File metadata

  • Download URL: duckle-0.7.1-py3-none-win_amd64.whl
  • Upload date:
  • Size: 27.6 MB
  • Tags: Python 3, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for duckle-0.7.1-py3-none-win_amd64.whl
Algorithm Hash digest
SHA256 74d717f9f63717fd936b7fadae8a93d114f94dd76ec5f46f92f74d7e2b06a64d
MD5 e5d726caeb9d586044bc14a64c1b25af
BLAKE2b-256 05a193a62eb62cf36dae6e7d75e0f3d9ee9532c2613d1f8191cc68eb087435ad

See more details on using hashes here.

Provenance

The following attestation bundles were made for duckle-0.7.1-py3-none-win_amd64.whl:

Publisher: pypi.yml on slothflowlabs/duckle

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file duckle-0.7.1-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for duckle-0.7.1-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 e8e0a1f1f98a3e228ca140def9d2e0365f66fe43d4b132563a0a3a111d6c86a9
MD5 f06263bab417f13610e70548e2b63bfb
BLAKE2b-256 8ea33a4f5a5dece5d7a3def0397d7fd1d18974503fc4a3bfe3fae9edb424082a

See more details on using hashes here.

Provenance

The following attestation bundles were made for duckle-0.7.1-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: pypi.yml on slothflowlabs/duckle

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file duckle-0.7.1-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl.

File metadata

File hashes

Hashes for duckle-0.7.1-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl
Algorithm Hash digest
SHA256 ba5a9b08bf21788597cd7f954dba7566f9c25fc0e49b6e3e1bec1845f3b88e58
MD5 e51c2443537341252ce187e050923f05
BLAKE2b-256 1b57c3b4dec3eb1af608b5d4f377da32a1df6eaec85420a711e898bd8f555959

See more details on using hashes here.

Provenance

The following attestation bundles were made for duckle-0.7.1-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl:

Publisher: pypi.yml on slothflowlabs/duckle

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file duckle-0.7.1-py3-none-macosx_11_0_x86_64.whl.

File metadata

File hashes

Hashes for duckle-0.7.1-py3-none-macosx_11_0_x86_64.whl
Algorithm Hash digest
SHA256 f14913dfb93ca12e37392c39feada52dd022b5b3b2e36dcadd7ec5b374bf508e
MD5 6aabd6abcc307ac09c2001aebc16e96e
BLAKE2b-256 ec2f114feb8a9db1edd5c3da056ff8f38cb60ce63a078d7cd6f64e66b2fbb82b

See more details on using hashes here.

Provenance

The following attestation bundles were made for duckle-0.7.1-py3-none-macosx_11_0_x86_64.whl:

Publisher: pypi.yml on slothflowlabs/duckle

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file duckle-0.7.1-py3-none-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for duckle-0.7.1-py3-none-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 5dd6da4beb7ee7c932129766e404a5294bbc8400eeb0a16c6469b1705b1f0657
MD5 813cc1548f78fe51baf5cab915a31ad2
BLAKE2b-256 3ff9e1458bf424a431b41355b98e26d8efb1268d6cd43d3de1c9eb1470f18e80

See more details on using hashes here.

Provenance

The following attestation bundles were made for duckle-0.7.1-py3-none-macosx_11_0_arm64.whl:

Publisher: pypi.yml on slothflowlabs/duckle

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.7.1 This release

6 files

0.5.11

6 files

0.5.10

6 files

0.5.9

6 files

0.5.8

6 files

0.5.7

6 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page