Skip to main content
Daffodil logo

Python Daffodil

Daffodil (DAtaFrames For Optimized Data Inspection and Logical processing) gives Python fast, lightweight 2-D data tables. A table is a list of rows, and each row is a plain Python list. Use it where data arrives and is processed row by row: reading files, building records, cleaning, reshaping, and writing them out again.

Documentation: https://raylutz.github.io/daffodil/

Status: alpha, and used in production for several years. The changes of each release are in the changelog.

Install

pip install daffodil

Python 3.10 or later. The table code is pure Python. pandas, NumPy and a few other packages are needed only for the features that use them, such as to_pandas_df(), and are not installed with daffodil. The Install section of the Guide lists them.

The data model

The Daffodil data model: lol, hd, kd and dtypes

The rows are kept in lol, a list of lists. The column names are kept once, in the header dict hd, which maps each name to its position. A table can have a keyfield, a column whose values identify the rows, and then the key dict kd maps each key to its row. A cell can hold any Python object: text, a number, a list, a dict, or even another table.

Quick start

from daffodil.daf import Daf

daf = Daf(cols=['id', 'item', 'qty'], keyfield='id')
daf.append({'id': 'a1', 'item': 'pen', 'qty': 3})
daf.append(['b2', 'ink', 10])

daf['b2']                     # the row with key 'b2'
daf[0]                        # the first row
daf[:, 'qty']                 # the qty column
daf['b2', 'qty'].to_value()   # 10
daf.select_where(lambda row: row['qty'] > 5)
print(daf)                    # a Markdown table

Why Daffodil

  • Adding a row is cheap. It is appended to a Python list. pandas has no cheap way to add a row, so code that uses it often collects the rows first and converts them at the end.
  • No conversion. The data stays the Python objects you put in, so there is no cost to get it in or out.
  • A selection shares the rows. Selecting rows copies no data, and lookups by key are dict lookups.
  • Mixed data is fine. A column can mix text, numbers and empty cells, and nothing is coerced to NaN or to a common type.
  • Tables print as Markdown, ready for a terminal or a report, and can be read back.
  • Small and pure Python, so it starts fast, which matters in short-lived cloud functions.

For heavy math over whole numeric columns, pandas, NumPy or Polars are faster. Build and clean the data as rows in Daffodil, then hand the numeric columns over with to_pandas_df() or to_numpy(). The documentation compares the two, with measurements.

Tables larger than memory

SqlDaf stores a table in a SQLite database file, so its rows stay out of Python memory. It supports tables larger than the memory available, and many operations run directly in SQLite. A small selection comes into memory as a Daf with to_daf().

A Daf in memory is faster. Whole-table operations ran 2 to 6 times faster in our timings, and a lookup by key about 11 times faster. So use a Daf for a table that fits in memory.

SqlDaf has the most common Daf methods, under the same names. Some behave differently, because the data is in a database. Any other Daf method raises NotImplementedError. The guide page Tables larger than memory shows how to use it.

Learn more

Questions and ideas are welcome as GitHub issues. Daffodil is released under the MIT license.

Metadata

Release files for daffodil 0.11.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for daffodil 0.11.0
File Size Uploaded Provenance Actions
daffodil-0.11.0.tar.gz 612.7 kB With provenance Details for daffodil-0.11.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for daffodil 0.11.0
File Interpreter ABI Platform Actions
daffodil-0.11.0-py3-none-any.whl Python 3 none any Details for daffodil-0.11.0-py3-none-any.whl

Total release size: 978.0 kB

Release files / daffodil-0.11.0.tar.gz

Download URL daffodil-0.11.0.tar.gz
Size 612.7 kB
Tags Source
SHA-256 checksum
How to use checksums
9476937803bce25bc942fda158e68da88e82b104e235eb4964fc3b0b065bb455
BLAKE2b-256 checksum
How to use checksums
555d0ca9c15f24e60a25df23b50422026d503a12f050825647be13885ccaec08
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 11, 2026.

Transparency log

Release files / daffodil-0.11.0-py3-none-any.whl

Download URL daffodil-0.11.0-py3-none-any.whl
Size 365.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
a13d51971a232665b4d07e466a974ab65643719ec081211e6f1094ec322c80f9
BLAKE2b-256 checksum
How to use checksums
6b38dee0f91cc7fade80b5310c32768f76dda538ccb701d0af64435f81e0f0b4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 11, 2026.

Transparency log

Security for daffodil 0.11.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for daffodil-0.11.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 11, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page