Skip to main content

datasuspy

CI PyPI DOI License: MIT

Fast reader for DATASUS .dbc files (and plain .dbf), returning Apache Arrow tables. The reader is written in Rust and shares its core with the datasusr R package's reader, against which it is tested value by value.

pip install datasuspy
import datasuspy

table = datasuspy.read_dbc("RDPE2401.dbc")        # pyarrow.Table
df = table.to_pandas()

import polars as pl
df = pl.from_arrow(datasuspy.read_dbc("RDPE2401.dbc", select=["uf_zi", "val_tot"]))

datasuspy only reads files that are already on disk. It does not download from the DATASUS FTP.

Options

read_dbc(path, **options) and open_dbc(path, **options) take:

Option Default Meaning
select None Columns to keep (case-insensitive). Unmatched names warn.
n_max None Maximum number of rows.
trim_ws True Trim whitespace from character fields.
encoding "latin1" "latin1" or "utf-8".
guess_types True Scan numeric fields to tell integers from floats.
col_types None {name: "character" | "integer" | "double" | "logical" | "date"}.
parse_dates False Return DBF date fields as dates instead of text.
clean_names True Lowercase the column names.
batch_size 65536 Rows per Arrow record batch.

open_dbc() returns a DbcReader: a one-shot Arrow stream implementing the PyCapsule interface (__arrow_c_stream__), so it can be consumed in batches by pyarrow.RecordBatchReader.from_stream, polars, DuckDB and others.

import pyarrow as pa
import pyarrow.parquet as pq

reader = pa.RecordBatchReader.from_stream(datasuspy.open_dbc("RDPE2401.dbc"))
with pq.ParquetWriter("rd.parquet", reader.schema) as writer:
    for batch in reader:
        writer.write_batch(batch)

Behaviour

  • Records flagged as deleted are dropped; n_max counts live records.
  • Character columns are never null: a blank field is an empty string.
  • Numeric fields with no decimals are int32 unless a value does not fit; then the column is float64. Unparseable values are null.
  • Date fields (D) are text unless parse_dates=True; invalid dates are null.
  • Latin-1 is decoded as strict ISO-8859-1.

Source, issues and the command-line converter: https://github.com/StrategicProjects/datasus-rs

MIT license. The decompressor is a translation of Mark Adler's blast.c (zlib license).

Metadata

Release files for datasuspy 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for datasuspy 0.1.0
File Size Uploaded Provenance Actions
datasuspy-0.1.0.tar.gz 36.4 kB With provenance Details for datasuspy-0.1.0.tar.gz

Built distributions (wheels)

Table of built distributions (wheels) for datasuspy 0.1.0
File Actions
datasuspy-0.1.0-cp310-abi3-win_amd64.whl CPython 3.10 abi3 Windows x86-64 Details for datasuspy-0.1.0-cp310-abi3-win_amd64.whl
datasuspy-0.1.0-cp310-abi3-musllinux_1_2_x86_64.whl CPython 3.10 abi3 Linux musl 1.2+ x86-64 Details for datasuspy-0.1.0-cp310-abi3-musllinux_1_2_x86_64.whl
datasuspy-0.1.0-cp310-abi3-musllinux_1_2_aarch64.whl CPython 3.10 abi3 Linux musl 1.2+ ARM64 Details for datasuspy-0.1.0-cp310-abi3-musllinux_1_2_aarch64.whl
datasuspy-0.1.0-cp310-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl CPython 3.10 abi3 Linux glibc 2.17+ x86-64 Details for datasuspy-0.1.0-cp310-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
datasuspy-0.1.0-cp310-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl CPython 3.10 abi3 Linux glibc 2.17+ ARM64 Details for datasuspy-0.1.0-cp310-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl
datasuspy-0.1.0-cp310-abi3-macosx_11_0_arm64.whl CPython 3.10 abi3 macOS 11.0+ ARM64 Details for datasuspy-0.1.0-cp310-abi3-macosx_11_0_arm64.whl
datasuspy-0.1.0-cp310-abi3-macosx_10_12_x86_64.whl CPython 3.10 abi3 macOS 10.12+ x86-64 Details for datasuspy-0.1.0-cp310-abi3-macosx_10_12_x86_64.whl

Total release size: 3.2 MB

Release files / datasuspy-0.1.0.tar.gz

Download URL datasuspy-0.1.0.tar.gz
Size 36.4 kB
Tags Source
SHA-256 checksum
How to use checksums
0ea46397712d3e330711a21d5dd0404e518f72d8aa96abc82af68ddd757fcdd5
BLAKE2b-256 checksum
How to use checksums
f81b27db3d5bf53c1518d8df0874cb5d817e827d769377beba087068890ed1fb
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.

Transparency log

Release files / datasuspy-0.1.0-cp310-abi3-win_amd64.whl

Download URL datasuspy-0.1.0-cp310-abi3-win_amd64.whl
Size 301.3 kB
Tags CPython 3.10 Windows x86-64 abi3
SHA-256 checksum
How to use checksums
a0974055974ff1c0e50c6f93bdb457504ae2cf9e30d0cdc387194d02ef9fe40f
BLAKE2b-256 checksum
How to use checksums
8fb0fa5cd82f99119e8b95a77a81e79c49e040e0ca1a997d6280106108d51e83
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.

Transparency log

Release files / datasuspy-0.1.0-cp310-abi3-musllinux_1_2_x86_64.whl

Download URL datasuspy-0.1.0-cp310-abi3-musllinux_1_2_x86_64.whl
Size 643.7 kB
Tags CPython 3.10 Linux musl 1.2+ x86-64 abi3
SHA-256 checksum
How to use checksums
7d9554bbe5c31be054424af7e7c28c17df3860749095fdaff275da1fde258b63
BLAKE2b-256 checksum
How to use checksums
272877201d0d49de032cd959414778097e8864c2cfee507fc2eef8983d633331
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.

Transparency log

Release files / datasuspy-0.1.0-cp310-abi3-musllinux_1_2_aarch64.whl

Download URL datasuspy-0.1.0-cp310-abi3-musllinux_1_2_aarch64.whl
Size 610.0 kB
Tags CPython 3.10 Linux musl 1.2+ ARM64 abi3
SHA-256 checksum
How to use checksums
88d61f1cfcb408bd748fa2b84570e7f4fb72e4b62aad01509bda14349dcff954
BLAKE2b-256 checksum
How to use checksums
80650228d0195bd1d0b1f531ab26e9c98d3aea98922811f7ffcd2129d7645566
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.

Transparency log

Release files / datasuspy-0.1.0-cp310-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl

Download URL datasuspy-0.1.0-cp310-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Size 431.3 kB
Tags CPython 3.10 Linux glibc 2.17+ x86-64 abi3
SHA-256 checksum
How to use checksums
3b21d3a144fc388f0a6f058fc27490b92e81c0c469d4089a4eee2c2cfb55466a
BLAKE2b-256 checksum
How to use checksums
f442d2abd66ec1cb546e56b6cce412a2faa796673daebc470503045394bfa808
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.

Transparency log

Release files / datasuspy-0.1.0-cp310-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl

Download URL datasuspy-0.1.0-cp310-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl
Size 432.1 kB
Tags CPython 3.10 Linux glibc 2.17+ ARM64 abi3
SHA-256 checksum
How to use checksums
d7e3e1006daa206457bea5b8380d7ea31134afd13f0351bcd8875142359571d2
BLAKE2b-256 checksum
How to use checksums
425fd6d20043854aafa6afbb5133480a1aac2b547e79bb887d6d0e6511621323
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.

Transparency log

Release files / datasuspy-0.1.0-cp310-abi3-macosx_11_0_arm64.whl

Download URL datasuspy-0.1.0-cp310-abi3-macosx_11_0_arm64.whl
Size 393.4 kB
Tags CPython 3.10 abi3 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
f1ad1bf87d5a0290e55ad01262536d33135fba10eaed22a431ef3411fbbac547
BLAKE2b-256 checksum
How to use checksums
3a8af1f342034494aa7f9154c0749249ac4cfd631732d38403e602340b8af86d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.

Transparency log

Release files / datasuspy-0.1.0-cp310-abi3-macosx_10_12_x86_64.whl

Download URL datasuspy-0.1.0-cp310-abi3-macosx_10_12_x86_64.whl
Size 398.5 kB
Tags CPython 3.10 abi3 macOS 10.12+ x86-64
SHA-256 checksum
How to use checksums
b5967e1a6f53d261bac3c887875c94037d704713547a0c293ee95cea31d257d9
BLAKE2b-256 checksum
How to use checksums
0f6e428dd9d9c955aead6351e3509f82b9097b6efbeacdd1e98db9cea493ebeb
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.

Transparency log

Security for datasuspy 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

All 8 files in this release have been uploaded with provenance metadata.

Select from 8 files

Attestation for datasuspy-0.1.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

8 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page