nbase-converter

nbase-converter is a native Rust/Arrow Python package for reading, writing, and converting SAS7BDAT, SAS XPORT v5/v8, and Parquet datasets.

Version 0.0.6 is the next alpha release, not yet published. The production SAS7BDAT writer's previous Windows SAS 9.4 validation is recorded in the repository release notes. Validation of the expanded encoding support remains pending; stable 1.0.0 compatibility is not yet declared.

Install

The command below applies after 0.0.6 is published. Before publication, install the wheel built from this checkout for local validation.

python -m pip install nbase-converter==0.0.6

Python 3.11 through 3.14 is supported on 64-bit Linux, macOS, and Windows.

Example

from nbase_converter import convert_dataset, filter_field, read_dataset, write_dataset

result = read_dataset(
    "input.xpt",
    columns=["USUBJID", "AGE"],
    filter=filter_field("AGE") >= 18,
    output="pandas",
)

write_dataset(
    result.data,
    "output.parquet",
    target_format="parquet",
    overwrite=True,
)

convert_dataset(
    "source.parquet",
    "submission.xpt",
    target_format="xpt8",
    metadata={
        "dataset_label": "Adverse Events",
        "columns": {
            "ASTDT": {
                "label": "Analysis Start Date",
                "sas": {"format": "IS8601DA", "length": 8},
            }
        },
    },
)

The public dataset operations are:

• read_dataset: materialized, streaming, or metadata-only reads with projection, row bounds, encoding selection, and structured filters.
• write_dataset: Arrow-compatible writes to Parquet, XPT v5/v8, or SAS7BDAT.
• convert_dataset: bounded-memory full-file conversion with optional supplemental metadata.

For writes and conversions, explicit metadata overrides recognized input metadata while omitted attributes remain preserved. Supported supplemental attributes are the dataset label and each column's label, SAS format, informat, and storage length. For string attributes, "" explicitly clears an inherited value; whitespace-only labels remain literal labels.

Text encoding

write_dataset(..., encoding=None) and convert_dataset(..., encoding=None) leave output encoding unspecified. The native engine selects UTF-8 when the option is omitted or None. XPT v5/v8 and SAS7BDAT also accept Windows-1252, ISO-8859-1, ASCII, Shift-JIS, and Big5. Names are case-insensitive; common aliases such as utf8, cp1252, latin1, us-ascii, sjis, cp932, and big-5 are accepted. ISO-8859-1 and ASCII use their literal character sets. Parquet output always uses UTF-8 and rejects other requested encodings. UTF-16 is unsupported for these dataset targets. Unrepresentable characters and byte-width overflows fail without replacing text, truncating values, or publishing a partial target.

read_dataset(encoding=...) selects input decoding. For conversion, input decoding is selected separately with source_encoding; existing conversion calls using encoding to decode the source must migrate to source_encoding. encoding now selects output encoding.

convert_dataset("input.parquet", "output.xpt", target_format="xpt8")
convert_dataset("input.parquet", "legacy.xpt", target_format="xpt8", encoding="Windows-1252")
convert_dataset(
    "legacy.xpt", "output.sas7bdat", target_format="sas7bdat",
    source_encoding="Windows-1252", encoding=None,
)

Reports contain the resolved output encoding in encodings.target, including UTF-8 when encoding=None. encodings.selected remains the source decoding override. XPT files do not reliably declare their encoding: recipients must select the matching reader or SAS session encoding. The package's default XPT reader retains UTF-8 decoding with Windows-1252 fallback; use an explicit input encoding for other legacy XPT encodings.

The R API uses encoding=NULL for output and source_encoding=NULL for conversion input. The CLI uses --encoding for output and --source-encoding for input; omitted output encoding is resolved by the engine to UTF-8.

Structured filters are created with filter_field, combined with filter_all, filter_any, or filter_not, and evaluated by these leaf helpers:

• comparisons: filter_equal, filter_not_equal, filter_less_than, filter_less_than_or_equal, filter_greater_than, and filter_greater_than_or_equal;
• collections and text: filter_in, filter_between, filter_contains, filter_starts_with, and filter_ends_with;
• missing values: filter_is_missing, filter_is_not_missing, filter_is_null, and filter_is_not_null.

Python comparison operators are also available on filter_field(...). Public result and metadata contracts ship as TypedDict definitions, and the wheel is marked as a typed package.

The default cross-format type policy accepts Float64 and UTF-8 text. Non-null IEEE NaNs are written as standard SAS numeric missing values and reported as coercions. Local filesystem paths are the only supported storage interface in this release.

Source, issues, release notes, and the full contract are available from the Bitbucket repository.