sas2parquet

The ultimate SAS (.sas7bdat) to Parquet converter — built to handle files that fail with standard tools.

sas2parquet automatically detects encodings, repairs schemas, infers correct data types, and performs pixel-perfect validation between SAS and Parquet outputs.

---

✨ Features

Feature	Description
🔄 Auto Encoding	Detects UTF-8, Latin1, CP1252 from metadata or fallback
🧠 Smart Types	Infers datetime, numeric, string with 20+ retry strategies
✅ Validation	Chunk-by-chunk comparison (metadata, counts, values)
📊 Memory Safe	Chunked processing (96GB RAM optimized, configurable)
💾 ZSTD Compression	Level-6 ZSTD for efficient Parquet storage
📝 Detailed Logs	Full conversion trace + mismatch reports
🎯 Two Modes	Single file or recursive directory processing

---

🚀 Quick Start

Install

pip install sas2parquet

---

✅ Usage

Convert a directory (recommended)

sas2parquet path/to/sasdata/

• Converts all .sas7bdat files recursively
• Creates parquetdata/ and logging/ next to sasdata/

---

Convert a single file

sas2parquet path/to/file.sas7bdat

Output (default):

path/to/file.parquet

---

Specify output location

Directory mode — custom output directory

sas2parquet path/to/sasdata/ --out path/to/parquetdata/

File mode — custom output file

sas2parquet path/to/file.sas7bdat --out path/to/output.parquet

---

Custom log directory (directory mode)

sas2parquet path/to/sasdata/ --log-dir path/to/logs/

---

📁 Directory Mode Behavior

your-project/
├── sasdata/
│   ├── file1.sas7bdat
│   └── subfolder/
│       └── nested.sas7bdat
├── parquetdata/
│   ├── file1.parquet
│   └── subfolder_parquet/
│       └── nested.parquet
└── logging/
    └── conversion_20260205_1145.log

---

🛠️ CLI Reference

sas2parquet --help

---

⚙️ Configuration (Advanced)

Edit constants in:

src/sas2parquet/convert.py

---

👥 Authors

This package was built collaboratively.

• Zaman Ziabakhshganji — creator and maintainer
• Anete Kristiana Zonnenberga — co-author and contributor
• Farshad Radman — co-author and contributor
• Jos van Dongen — co-author and contributor

📄 License

MIT License