SciDuck
The DuckDB layer under scidb. It owns two things: the codec that turns
Python values into queryable DuckDB columns and back, and a locked
connection that is safe to share across threads. It creates no tables:
scidb owns every table of a project database (scidb.store.tables) and the
schema-location ids (scidb.store.sciduck.SciDuck, moved out of sciduckdb in P3).
Application code reaches the database only through scidb.
All data, arrays included, is stored in queryable DuckDB types (LIST, nested LIST, JSON), so the database can be inspected with DBeaver or any DuckDB-compatible viewer.
Codec
from sciduckdb import infer_data_columns, value_to_storage_row, restore_types, storage_to_python
col_types, dtype_meta = infer_data_columns(value) # {column: DuckDB type}, restore metadata
row = value_to_storage_row(value, dtype_meta) # one row of storage values
# ... after reading the rows back into a DataFrame df:
df = restore_types(df, dtype_meta) # Python types restored per column
- Scalars, numpy arrays (any dtype and shape), lists, and dicts (flat or nested; nested keys become dot-separated columns) store one row per value.
- A DataFrame value stores one DuckDB row per DataFrame row
(
dataframe_to_storage_rows), one column per DataFrame column. dtype_metarecords what each column was, so a load is lossless: dtypes, shapes and NULL list elements (restored as NaN) survive the round trip.
Connection
DuckConnection(path) opens the file. Every statement runs
under one lock, so a concurrent close() cannot land mid-statement. Group
paired work with with duck.transaction("label"):; it is re-entrant on the
owning thread and rolls back on any exception.
The supported interface of a connection (DuckConnection, which scidb's
SciDuck extends). Other packages call these and nothing with a leading underscore:
| Method | Does |
|---|---|
transaction(label) |
context manager: one transaction around the block |
execute(sql, params) |
a statement that returns no rows; returns None |
fetchall / fetchone |
rows as tuples (few rows) / the first row or None |
fetchdf |
rows as a DataFrame (many rows; NULL floats become NaN) |
fetch_table |
(columns, rows) with NULL kept as None (diagnostics) |
fetchdf_if_open |
fetchdf, or None when the connection is closed |
table_exists(name) |
whether a table or view exists |
bulk_insert / bulk_update |
vectorized multi-row writes (one statement) |
add_rollback_listener(fn) |
call fn() after every outermost ROLLBACK |
query(sql) |
fetchdf under its user-facing name |
is_open, close(), reopen() |
connection lifetime; also a context manager |
Every fetch holds the lock across execute and fetch. execute returns
nothing so a result can never be fetched after the lock is released.
Modules: codec.py (conversion functions), connection.py
(DuckConnection), datetimes.py, row_lists.py. Import from sciduckdb, not from the modules.
Schema ids
Moved to scidb (scidb.store.sciduck.SciDuck): _schema, _variables and
_variable_groups are project tables like any other.
Datetimes
A datetime value (datetime.datetime, pd.Timestamp, np.datetime64, NaT) is
stored as two columns: <col> TIMESTAMP_NS holding the UTC instant (or a
naive value's wall clock), and <col>__tz VARCHAR holding that value's zone
("Europe/Berlin", "UTC", "+02:00"; NULL = naive). Zones are per value, so
one column may hold several zones and naive values. Loads give back each value
in its own zone. The __tz column exists from the column's first save and is
never listed in the variable's data columns. In raw SQL, filter on
<col>__tz IS NULL to tell naive rows apart. Owner: sciduckdb.datetimes;
design: docs/claude/datetime-roundtrip.md.
Metadata
Release files for sciduckdb 0.2.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| sciduckdb-0.2.2.tar.gz | 32.5 kB | With provenance | Details for sciduckdb-0.2.2.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| sciduckdb-0.2.2-py3-none-any.whl | Python 3 | none | any | Details for sciduckdb-0.2.2-py3-none-any.whl |
Total release size: 66.6 kB
Release files / sciduckdb-0.2.2.tar.gz
| Download URL | sciduckdb-0.2.2.tar.gz |
|---|---|
| Size | 32.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5637fc8a4a20411b090f3c41151b5463a9c0619d0fe6a4633e18e2f33aa85c28
|
|
BLAKE2b-256 checksum How to use checksums |
ee96668d5f2c01e9acc931be658677d449b67a48c4935a1884b4e24bded0d9ca
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.
Transparency logRelease files / sciduckdb-0.2.2-py3-none-any.whl
| Download URL | sciduckdb-0.2.2-py3-none-any.whl |
|---|---|
| Size | 34.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
388bdf9d934dc7dfde9f796cf6ee54485a3e97aa571ac5f87b068c5cf9a7f2b1
|
|
BLAKE2b-256 checksum How to use checksums |
da1a13ce7a5f91e9deb65809ac952035bc945fef0c5eb8df7df959e814d0b0fe
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.
Transparency logSecurity for sciduckdb 0.2.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for sciduckdb-0.2.2.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 3, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.