Distributed Dataframes for Multimodal Data

Navigation

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for desmondcheongzx from gravatar.com desmondcheongzx Avatar for madvart from gravatar.com madvart Avatar for robconhow from gravatar.com robconhow Avatar for samster25 from gravatar.com samster25

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • Author: Eventual Inc
  • Maintainer: Sammy Sidhu
  • Requires: Python >=3.10
  • Provides-Extra: all , audio , aws , azure , clickhouse , deltalake , gcp , google , gravitino , hudi , huggingface , iceberg , lance , numpy , openai , pandas , postgres , ray , sql , transformers , turbopuffer , unity , video , viz
Report project as malware

Project description

Daft dataframes can load any data such as PDF documents, images, protobufs, csv, parquet and audio files into a table dataframe structure for easy querying

GitHub Actions tests PyPI latest tag Coverage slack community

Website • Docs • Installation • Daft Quickstart • Community and Support

Daft: High-Performance Data Engine for AI and Multimodal Workloads

Eventual-Inc/Daft | Trendshift

Daft is a high-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale.

  • Native multimodal processing: Process images, audio, video, and embeddings alongside structured data in a single framework

  • Built-in AI operations: Run LLM prompts, generate embeddings, and classify data at scale using OpenAI, Transformers, or custom models

  • Python-native, Rust-powered: Skip the JVM complexity with Python at its core and Rust under the hood for blazing performance

  • Seamless scaling: Start local, scale to distributed clusters on Ray, Kubernetes, or Daft Cloud

  • Universal connectivity: Access data anywhere (S3, GCS, Iceberg, Delta Lake, Hugging Face, Unity Catalog)

  • Out-of-box reliability: Intelligent memory management and sensible defaults eliminate configuration headaches

Getting Started

Installation

Install Daft with pip install daft. Requires Python 3.10 or higher.

For more advanced installations (e.g. installing from source or with extra dependencies such as Ray and AWS utilities), please see our Installation Guide

Quickstart

Get started in minutes with our Quickstart - load a real-world e-commerce dataset, process product images, and run AI inference at scale.

More Resources

  • Examples - see Daft in action with use cases across text, images, audio, and more

  • User Guide - take a deep-dive into each topic within Daft

  • API Reference - API reference for public classes/functions of Daft

Benchmarks

AI Benchmarks

To see the full benchmarks, detailed setup, and logs, check out our benchmarking page.

Contributing

We ❤️ developers! To start contributing to Daft, please read CONTRIBUTING.md. This document describes the development lifecycle and toolchain for working on Daft. It also details how to add new functionality to the core engine and expose it through a Python API.

Here’s a list of good first issues to get yourself warmed up with Daft. Comment in the issue to pick it up, and feel free to ask any questions!

Telemetry

To help improve Daft, we collect non-identifiable data via Scarf (https://scarf.sh).

To disable this behavior, set the environment variable DO_NOT_TRACK=true.

The data that we collect is:

  1. Non-identifiable: Events are keyed by a session ID which is generated on import of Daft

  2. Metadata-only: We do not collect any of our users’ proprietary code or data

  3. For development only: We do not buy or sell any user data

Please see our documentation for more details.

https://static.scarf.sh/a.png?x-pxid=31f8d5ba-7e09-4d75-8895-5252bbf06cf6

License

Daft has an Apache 2.0 license - please see the LICENSE file.

Project details

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for desmondcheongzx from gravatar.com desmondcheongzx Avatar for madvart from gravatar.com madvart Avatar for robconhow from gravatar.com robconhow Avatar for samster25 from gravatar.com samster25

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • Author: Eventual Inc
  • Maintainer: Sammy Sidhu
  • Requires: Python >=3.10
  • Provides-Extra: all , audio , aws , azure , clickhouse , deltalake , gcp , google , gravitino , hudi , huggingface , iceberg , lance , numpy , openai , pandas , postgres , ray , sql , transformers , turbopuffer , unity , video , viz

Release history Release notifications | RSS feed

0.7.13

0.7.12

0.7.11 yanked

Reason this release was yanked:

Breaking change in extensions

0.7.10

0.7.9

0.7.8

0.7.7

0.7.6

This version

0.7.5

0.7.4

0.7.3

0.7.2

0.7.1

0.7.0

0.6.14

0.6.13

0.6.12

0.6.11

0.6.10

0.6.8

0.6.7

0.6.6

0.6.5

0.6.4

0.6.3

0.6.2

0.6.1

0.6.0

0.5.22

0.5.21

0.5.20

0.5.19

0.5.18

0.5.17

0.5.16

0.5.15

0.5.14

0.5.13

0.5.12

0.5.11

0.5.10

0.5.9

0.5.8

0.5.7

0.5.6

0.5.5

0.5.4

0.5.3

0.5.2

0.5.1

0.5.0

0.4.18

0.4.17

0.4.16

0.4.15

0.4.14

0.4.13

0.4.12

0.4.11

0.4.10

0.1.3 yanked

Reason this release was yanked:

This project has migrated to `daft-pgm`. Please run `pip install daft-pgm` instead

0.1.2 yanked

Reason this release was yanked:

This project has migrated to `daft-pgm`. Please run `pip install daft-pgm` instead

0.1.0 yanked

Reason this release was yanked:

This project has migrated to `daft-pgm`. Please run `pip install daft-pgm` instead

0.0.4 yanked

Reason this release was yanked:

This project has migrated to `daft-pgm`. Please run `pip install daft-pgm` instead

0.0.3 yanked

Reason this release was yanked:

This project has migrated to `daft-pgm`. Please run `pip install daft-pgm` instead

0.0.2 yanked

Reason this release was yanked:

This project has migrated to `daft-pgm`. Please run `pip install daft-pgm` instead

0.0.1 yanked

Reason this release was yanked:

This project has migrated to `daft-pgm`. Please run `pip install daft-pgm` instead

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

daft-0.7.5.tar.gz (2.8 MB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

daft-0.7.5-cp310-abi3-win_amd64.whl (57.5 MB view details)

Uploaded CPython 3.10+Windows x86-64

daft-0.7.5-cp310-abi3-manylinux_2_24_x86_64.whl (56.7 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.24+ x86-64

daft-0.7.5-cp310-abi3-manylinux_2_24_aarch64.whl (54.6 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.24+ ARM64

daft-0.7.5-cp310-abi3-macosx_11_0_arm64.whl (52.2 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

daft-0.7.5-cp310-abi3-macosx_10_12_x86_64.whl (56.3 MB view details)

Uploaded CPython 3.10+macOS 10.12+ x86-64

File details

Details for the file daft-0.7.5.tar.gz.

File metadata

  • Download URL: daft-0.7.5.tar.gz
  • Upload date:
  • Size: 2.8 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for daft-0.7.5.tar.gz
Algorithm Hash digest
SHA256 d04ee44185f280dd2735990edf01f3a60b7dde86870d4d359602a73081f08314
MD5 b662a79a3ae754b2d01570a636320bf2
BLAKE2b-256 5f13683623b5e62ffb50c761f14368e18a7caaab5993b7597db8f75d4ea036b2

See more details on using hashes here.

Provenance

The following attestation bundles were made for daft-0.7.5.tar.gz:

Publisher: publish-pypi.yml on Eventual-Inc/Daft

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file daft-0.7.5-cp310-abi3-win_amd64.whl.

File metadata

  • Download URL: daft-0.7.5-cp310-abi3-win_amd64.whl
  • Upload date:
  • Size: 57.5 MB
  • Tags: CPython 3.10+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for daft-0.7.5-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 eb7b6da90fc1af2575d3a6107425db27373d14622134136f4926239db1422633
MD5 87e38ff74dedb5268ae6a4245aa8297e
BLAKE2b-256 47f7d371d4f48818341557d9c3fad75bea5970d2f8e3b95016d13761297d2218

See more details on using hashes here.

Provenance

The following attestation bundles were made for daft-0.7.5-cp310-abi3-win_amd64.whl:

Publisher: publish-pypi.yml on Eventual-Inc/Daft

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file daft-0.7.5-cp310-abi3-manylinux_2_24_x86_64.whl.

File metadata

File hashes

Hashes for daft-0.7.5-cp310-abi3-manylinux_2_24_x86_64.whl
Algorithm Hash digest
SHA256 cfb44d540e4e32d99c4281f69fb6c071a168bca98fcea93bd0a36e611b055e58
MD5 ccd2123b8c3e8b47a17d9d1162a161b6
BLAKE2b-256 85a250c137f21f30b09674774204bed1974dda7e1be9eb78da7badf969aba408

See more details on using hashes here.

Provenance

The following attestation bundles were made for daft-0.7.5-cp310-abi3-manylinux_2_24_x86_64.whl:

Publisher: publish-pypi.yml on Eventual-Inc/Daft

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file daft-0.7.5-cp310-abi3-manylinux_2_24_aarch64.whl.

File metadata

File hashes

Hashes for daft-0.7.5-cp310-abi3-manylinux_2_24_aarch64.whl
Algorithm Hash digest
SHA256 2a2fad56e2d941d91bdb8ae1692b4e96f1fc151addb15427b67cdba5f561d36d
MD5 5c98d3febcbe267f94b94713c683684d
BLAKE2b-256 260d0e508bb591a26626ca689e556a43cac947b40b10133021c8276d9acb96b8

See more details on using hashes here.

Provenance

The following attestation bundles were made for daft-0.7.5-cp310-abi3-manylinux_2_24_aarch64.whl:

Publisher: publish-pypi.yml on Eventual-Inc/Daft

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file daft-0.7.5-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

  • Download URL: daft-0.7.5-cp310-abi3-macosx_11_0_arm64.whl
  • Upload date:
  • Size: 52.2 MB
  • Tags: CPython 3.10+, macOS 11.0+ ARM64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for daft-0.7.5-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 50d50ef987d5a7d75b2db50c03f0de3e597115ad1c9c461ae4a9799e7fbdf783
MD5 f2869f01f0b599a5ad873b2edd714ce8
BLAKE2b-256 3b44703f39c5a15e0409134f4959b848f790da7412f66bad1dceb6c9734c1a05

See more details on using hashes here.

Provenance

The following attestation bundles were made for daft-0.7.5-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: publish-pypi.yml on Eventual-Inc/Daft

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file daft-0.7.5-cp310-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for daft-0.7.5-cp310-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 b7bbe1cfa3dcef3128443f73ccc00c37465641ab0bbf5f2bb85311eac42ef2cc
MD5 abdd67dc1d7bc019cbfb67cf81ffe36b
BLAKE2b-256 20de8a93c014febdabd0dff987dd12a7bf69c9fa795bbe9a5c7998959b7b2775

See more details on using hashes here.

Provenance

The following attestation bundles were made for daft-0.7.5-cp310-abi3-macosx_10_12_x86_64.whl:

Publisher: publish-pypi.yml on Eventual-Inc/Daft

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page