VANE

A high-performance, multimodal-native engine for AI workloads

PyPI Apache License 2.0 Ask DeepWiki

Join Discord Follow AstroVelaAI on X

Vane unifies multimodal data, intelligence, and continuous learning with Python and SQL interfaces, seamlessly scaling from local environments to Ray clusters.

Vane platform overview

[!NOTE] Project status

  • Vane Data — Supports most of the capabilities described below and is under active development, but is not yet production-ready. Its interfaces and internals may continue to evolve as the codebase is reviewed and hardened.
  • Vane RL and Vane Agent — In the early stages of design and implementation. Their source code will be released in future updates.
  • Vibe Coding and Agentic Engineering — Some parts of our system were initially built through Vibe Coding. We are now continuously analyzing, understanding, and improving the codebase, applying an Agentic Engineering approach to drive iterative optimization and enhance the quality, maintainability, and efficiency of the system.

Vane Data

Vane Data is a high-performance, multimodal-native data engine for AI workloads. Built on a fork of DuckDB, it extends the core execution engine with native multimodal processing and a unified framework for local and distributed execution.

Vane Data architecture

Key Features

  • Multimodal-native processing — Process images, video, audio, text, documents, events, sensor data, and tables through a unified type system. Dynamic batching and backpressure control handle variations in data size and computational cost.
  • Python and SQL interfaces — Build data and AI pipelines with DuckDB SQL or the Python Relation API.
  • Built-in AI operations — Invoke LLMs, generate embeddings, and run batch inference through OpenAI and Anthropic APIs or native vLLM integration. Prefix-aware bucketing improves vLLM prefix-cache hit rates and inference throughput.
  • Heterogeneous execution — Overlap CPU, GPU, I/O, and model inference workloads through asynchronous scheduling.
  • Local-to-cloud execution — Run the same pipeline locally or across distributed Ray clusters, with a foundation for future edge-cloud coordination.
  • Designed for production AI workloads — Build multimodal training-data preprocessing pipelines and enterprise-scale batch inference workflows.

Getting Started

Installation

Vane supports Python 3.10 through 3.14. Python 3.12 is recommended and is the primary development version.

Install the vane-ai package from PyPI:

pip install vane-ai

Vane owns only the vane Python namespace. It does not install duckdb, _duckdb, or adbc_driver_duckdb, so the official duckdb distribution can be installed in the same environment and both engines can be imported in the same process. Vane code must use import vane; import duckdb always refers to the separately installed official package. Vane does not provide a legacy duckdb alias or fall back to an official DuckDB native module.

import duckdb
import vane

assert vane.connect().execute("SELECT 42").fetchone() == (42,)
assert duckdb.connect().execute("SELECT 43").fetchone() == (43,)

Vane's ADBC driver is exposed as vane.adbc; the official driver's adbc_driver_duckdb namespace remains owned by the official distribution. Install adbc-driver-manager (also included by vane-ai[all]) to use either ADBC facade.

Optional features are provided as extras:

pip install 'vane-ai[openai]'   # OpenAI provider (anthropic / google / transformers / vllm likewise)
pip install 'vane-ai[image]'    # ndarray image inputs for AI providers (Pillow)
pip install 'vane-ai[video]'    # video data source (Pillow, psutil, decord)

The video extra installs decord on Linux x86-64, Vane's currently supported native platform. decord itself publishes no wheels for modern Python on macOS or for any ARM platform; if Vane adds Windows support later, decord's existing win_amd64 wheel can be enabled explicitly.

For more details, see the Installation Guide.

Quick Start

Follow the Quickstart guide to build and run your first Vane pipeline.

Execution Policy

Vane uses the Ray runner by default. If no runner is configured, executing a lazy relation through consumers such as display, result fetching, or file writes selects Ray and may lazily initialize it. An experimental local runner can be selected explicitly before creating connections:

import vane

vane.configure(runner="local")

Distributed Flight Transport

Vane follows Ray's trusted-cluster model: the driver, workers, submitted code, and east-west network belong to one trusted computing boundary. Same-process local-disk shuffle reads directly from the process-local registry, and object-storage shuffle reads committed manifests. Only cross-worker local-disk shuffle uses Arrow Flight.

A worker lazily starts one process-owned plaintext grpc:// Flight service when a local-disk exchange sink first needs it. The service provides no TLS, client authentication, query-level authorization, or tenant isolation. Keep its port reachable only inside the controlled Ray cluster network; workloads that do not trust one another require separate isolated Ray clusters.

Workers advertise their Ray private address by default. VANE_FLIGHT_BIND_HOST may select a different local bind address, including 0.0.0.0 in a container with appropriate network policy, while VANE_FLIGHT_ADVERTISE_HOST must always be a routable non-wildcard address. The advertised-host override is worker-local: set it in each worker node's environment rather than on the driver or in a Ray Job/actor runtime environment. DUCKDB_FLIGHT_PORT selects a fixed worker-local port; the default 0 lets the operating system allocate one. See SECURITY.md for the complete trust boundary.

Cross-worker reads have a one-hour call deadline and a 60-second maximum duration for each blocking DoGet, schema, or batch-read operation by default. Override them with VANE_FLIGHT_CALL_TIMEOUT_S and VANE_FLIGHT_READ_TIMEOUT_S; either value may be set to 0 to disable that timeout. The read timeout measures the complete Arrow operation, not byte-level network idleness. Query interruption also cancels an in-flight Flight call, so stalled consumers release the producer-side stream and its shuffle-file read lease.

More Resources

Multimodal Inference Benchmarks

Hardware configuration: 1 node, 36 CPU cores, 64 GB memory, and 1× NVIDIA GeForce RTX 2080 Ti (22 GB VRAM).

We use the Ray Data benchmark suite to compare Vane with Ray Data and Daft. The benchmark source code is included in this repository.

Multimodal inference benchmark comparing Vane Data, Ray Data, and Daft

The Ray runner targets distributed workloads. The current results are single-node only; validation on the multi-node environments used in the Ray Data benchmarks is still pending.

See the benchmarking page for detailed results.

Contributing

Contributions and collaborations are welcome. Contribution guidelines and community channels will be published as the project opens further.

License

Vane is distributed under the Apache License 2.0. See LICENSE and NOTICE for details and third-party attributions.

Acknowledgements

Vane Data is built on top of DuckDB and inspired by infrastructure systems such as Ray Data, Daft, and Trino.

  • DuckDB: The core modular architecture and inspiration. A high-performance analytical database system. It is designed to be fast, reliable, portable, and easy to use.
  • DuckDB-Python: The core modular architecture and inspiration. The DuckDB Python package.
  • Ray Data: A scalable data processing library for AI workloads built on Ray
  • Daft: High-Performance Data Engine for AI and Multimodal Workloads
  • Trino: A fast distributed SQL query engine for big data analytics.

Special thanks to these projects.

Give Vane a ⭐️ if it helps you!

Project links

Data verified by PyPI on Aug 11, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for kaka11chen from gravatar.com kaka11chen

Credits

Author: Vane contributors

Maintainer: Vane maintainers

GitHub Statistics

Data verified by PyPI on Aug 11, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

Apache-2.0
View SPDX License List

Requires

Python <3.15, >=3.10

Provides Extra

openai anthropic google transformers vllm image video all

Tags

Vane DuckDB Ray Distributed Database SQL OLAP

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

vane_ai-0.1.0.tar.gz (13.7 MB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

vane_ai-0.1.0-cp314-cp314-manylinux_2_28_x86_64.whl (37.1 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.28+ x86-64

vane_ai-0.1.0-cp313-cp313-manylinux_2_28_x86_64.whl (37.1 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.28+ x86-64

vane_ai-0.1.0-cp312-cp312-manylinux_2_28_x86_64.whl (37.1 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.28+ x86-64

vane_ai-0.1.0-cp311-cp311-manylinux_2_28_x86_64.whl (37.1 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.28+ x86-64

vane_ai-0.1.0-cp310-cp310-manylinux_2_28_x86_64.whl (37.1 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.28+ x86-64

File details

Details for the file vane_ai-0.1.0.tar.gz.

File metadata

  • Download URL: vane_ai-0.1.0.tar.gz
  • Upload date:
  • Size: 13.7 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for vane_ai-0.1.0.tar.gz
Algorithm Hash digest
SHA256 41de1e1ea2ed7e6ca2d0a1ac83d1d458feb02de099b9c8dd97d78f5d9100d803
MD5 5a6e873e1bfbee7c05497705213efcea
BLAKE2b-256 6a38f27aa9918198fb4050a0f6b2a83235fddf5a4239e8f7ba44ee2cd95f47cd

See more details on using hashes here.

Provenance

The following attestation bundles were made for vane_ai-0.1.0.tar.gz:

Publisher: release.yml on AstroVela/vane

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file vane_ai-0.1.0-cp314-cp314-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for vane_ai-0.1.0-cp314-cp314-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 11985d6c75e974fd35a9e89d0131af3950b9f62c983a27c2df827c8fff69e419
MD5 bb32ff0128925c66ad6d731b0a78e31c
BLAKE2b-256 6a4e87ecdee2845b6bdd8845e86bd597588261d9e719ae346e669c0770e8fc3b

See more details on using hashes here.

Provenance

The following attestation bundles were made for vane_ai-0.1.0-cp314-cp314-manylinux_2_28_x86_64.whl:

Publisher: release.yml on AstroVela/vane

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file vane_ai-0.1.0-cp313-cp313-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for vane_ai-0.1.0-cp313-cp313-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 131f5bb4dac26096f06dbaecb110ced149dd808b6eee0b139024dc8701e27903
MD5 72d0c5762bbc78009dec1eca68f3119f
BLAKE2b-256 c3e7e0276db83768269d9608a641490b5f86c87a87965ee8d5174ccda00bf601

See more details on using hashes here.

Provenance

The following attestation bundles were made for vane_ai-0.1.0-cp313-cp313-manylinux_2_28_x86_64.whl:

Publisher: release.yml on AstroVela/vane

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file vane_ai-0.1.0-cp312-cp312-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for vane_ai-0.1.0-cp312-cp312-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 1bbfa4cf0a0df9a23ea0a4e24a2ef5df9714133ebbde0f5cfb83280034054e0d
MD5 bde9dbf70d32a5be767c696d7c5f54fb
BLAKE2b-256 bd292f893c657220f0a9c9697ba17cc1c1c98410c02238b425e96bd50149e9c9

See more details on using hashes here.

Provenance

The following attestation bundles were made for vane_ai-0.1.0-cp312-cp312-manylinux_2_28_x86_64.whl:

Publisher: release.yml on AstroVela/vane

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file vane_ai-0.1.0-cp311-cp311-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for vane_ai-0.1.0-cp311-cp311-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 7f72fac20f70543f67addd52df68183d6bac95b923d45d27ef24b5c049af2353
MD5 d957c58a611443ca7db82205bbd45b18
BLAKE2b-256 80794b2732ffa120b67332e7567c273daaa1e0bf56e1b905b97391073b5720b5

See more details on using hashes here.

Provenance

The following attestation bundles were made for vane_ai-0.1.0-cp311-cp311-manylinux_2_28_x86_64.whl:

Publisher: release.yml on AstroVela/vane

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file vane_ai-0.1.0-cp310-cp310-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for vane_ai-0.1.0-cp310-cp310-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 5479b000f7a51c7c3902629fdfa5c36ff253f4ce6bfd1519a97303b32a4912b0
MD5 4c6921c653fb2e3c64318a1032bd816b
BLAKE2b-256 16465997c742d70aad6565ecf74490c6309d26f05ea760211780a413518e35b1

See more details on using hashes here.

Provenance

The following attestation bundles were made for vane_ai-0.1.0-cp310-cp310-manylinux_2_28_x86_64.whl:

Publisher: release.yml on AstroVela/vane

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page