duckless 0.3.1

Documentation · Quickstart · Benchmarks · Releases · Contributing

Serverless DuckDB on GCP. Submit SQL or your own code; DuckLess runs it in your project on a right-sized machine (Cloud Batch VM with local SSD for spill, or Cloud Run Jobs for small jobs), reads and writes GCS with the job's own credentials (no HMAC keys), then tears it down. Nothing runs between jobs. Optional DuckLake tables on a Cloud SQL catalog, and Agent Skills for coding agents.

Status: v0.3, young and tested on real projects. Changes are listed in the GitHub Releases; the benchmarks have the measurements behind the design.

Quick start

uv tool install duckless                     # or: pip install duckless

duckless init --project my-project --region europe-west1
# prints the lines to put in .envrc (DUCKLESS_PROJECT, DUCKLESS_BUCKET, DUCKLESS_SA, DUCKLESS_IMAGE)

duckless preflight --machine n2-highmem-32 --spot
duckless run job.sql --machine n2-highmem-32 --spot
duckless exec --image <your-image> --machine n2-highmem-16 -- dbt build
duckless status <job-id>
duckless logs <job-id> --follow
duckless result <job-id>
duckless destroy --project my-project       # removes what init created

init applies the Terraform module shipped with the CLI (duckless/terraform) through Infrastructure Manager: no local Terraform, state kept in your project, re-run it to upgrade. Teams managing infra as code can use the same module directly instead.

Full guide: tosun-si.github.io/duckless, from the quickstart to the CLI reference.

Writing jobs

• SQL (.sql): ${VAR} placeholders come from the job env (DUCKLESS_BUCKET, --env K=V).
• Python (.py): from duckless_runtime import connect gives a DuckDB connection with GCS auth, spill on local SSD, memory and threads sized to the VM.
• Write to GCS in parallel: COPY … TO 'gs://…' (FORMAT parquet, PER_THREAD_OUTPUT, FILE_SIZE_BYTES '256MB') is 7-8x faster than the single-writer default (~900 MB/s vs ~110 MB/s on 32 vCPU).
• Vectorize Python logic: a row-wise Python UDF runs at ~8k rows/s on one thread; use an Arrow UDF over numpy (type="arrow") or SQL.

More in Writing jobs and Machines, Spot and spill.

Agent Skills

DuckLess ships a Claude Code plugin with five skills: setup, writing-jobs, sizing, troubleshooting and ducklake. They carry what the docs and the CLI cannot decide for you (which machine, why a job failed) and the rules measured while building DuckLess.

duckless skills install          # into this project: .claude/skills + .agents/skills (--user: your home)

Or as a Claude Code plugin, kept up to date from this repository:

/plugin marketplace add tosun-si/duckless
/plugin install duckless@duckless

Layout

Hexagonal, kept light: a pure core, ports, adapters, and one wiring point.

Dependency rule: core imports nothing else from DuckLess, service only core and ports, and only wiring imports adapters.

Contributing

Issues and pull requests are welcome: bug reports, real use cases, docs, code. See CONTRIBUTING.md for the setup, the conventions and how changes are tested.

uv sync
uv run pytest
uv run ruff check . && uv run ruff format --check .

License

DuckLess is open source under the Apache License 2.0. You can use it, modify it and ship it, commercially or not; keep the license and the notices.