tabular-profiler 0.1.0

Profile CSV and Parquet files from the command line: inferred types, suggested dtypes, null/unique counts, min/max, and data-quality checks.

tabular-profiler

A small command-line tool that profiles a CSV, TSV, or Parquet file and tells you what is actually in each column — the inferred type, a suggested pandas dtype, null and unique counts, min/max — and flags common data problems such as columns that mix types or dates stored as plain strings.

$ profiler data.csv

Built with Click, pandas, and PyArrow.

---

Installation

Once published to PyPI:

pip install tabular-profiler

This installs a profiler command on your PATH.

To install from a checkout of this repository:

pip install .
# or, for development (editable install with test/build tooling):
pip install -e ".[dev]"

Requires Python 3.9+.

Usage

Point it at a file:

$ profiler examples/sample.csv
File:    examples/sample.csv
Format:  csv
Rows:    8
Columns: 6

column       inferred  stored  suggested       nulls    unique  min                  max
-----------  --------  ------  --------------  -------  ------  -------------------  -------------------
id           integer   int64   int8            0 (0%)   8       1                    8
signup_date  datetime  str     datetime64[ns]  0 (0%)   8       2021-01-15 00:00:00  2021-08-09 00:00:00
price        mixed     str     string          0 (0%)   8       15.00                oops
country      string    str     category        1 (12%)  3       CA                   US
notes        string    str     string          0 (0%)   8         welcome            regular
active       boolean   bool    bool            0 (0%)   2       False                True

Data quality findings:
  - [id] every value is unique; looks like an identifier
  - [signup_date] dates stored as text; suggest parsing to datetime64[ns]
  - [price] mixes multiple value types (float: 7, string: 1)
  - [notes] every value is unique; looks like an identifier
  - [notes] 4 value(s) have leading/trailing whitespace

Options

Option	Description
-f, --format [table|json]	Output format (default: table).
-n, --sample N	Only read the first N rows — handy for very large files.
-o, --output FILE	Write the report to a file instead of stdout.
--version	Show the version and exit.
-h, --help	Show help and exit.

JSON output

--format json emits a machine-readable report you can feed into other tools:

$ profiler data.csv --format json
{
  "path": "data.csv",
  "file_format": "csv",
  "n_rows": 8,
  "n_columns": 6,
  "columns": [
    {
      "name": "price",
      "stored_dtype": "str",
      "inferred_type": "mixed",
      "suggested_dtype": "string",
      "count": 8,
      "null_count": 0,
      "null_pct": 0.0,
      "unique_count": 8,
      "unique_pct": 100.0,
      "min": "15.00",
      "max": "oops",
      "issues": ["mixes multiple value types (float: 7, string: 1)"]
    }
  ],
  "dataset_issues": []
}

What it reports

For every column:

• inferred type — the logical type worked out from the values (integer, float, boolean, datetime, string, mixed, or empty), which can differ from how the data was stored.
• stored — the dtype pandas used to load the column.
• suggested — a recommended dtype. Suggestions aim to be both correct and memory-efficient:
  • the narrowest signed integer width that fits (int8 … int64);
  • Int64 (nullable) for integer columns with missing values, and for float columns whose values are all whole numbers;
  • datetime64[ns] for dates that were stored as text;
  • category for low-cardinality string columns;
  • string otherwise.
• nulls — count and percentage of missing values.
• unique — number of distinct non-null values.
• min / max — numeric/chronological for numbers and dates, alphabetical for text.

Data-quality checks

Per column:

• Mixed types — values that don't agree on a type (e.g. mostly numbers with a few stray strings). Integers and floats together are not flagged — that's just a float column.
• Dates stored as text — text columns whose values parse as dates.
• Numbers stored as text — text columns whose values are all numeric.
• High null fraction — at least half the values are missing (entirely empty columns are called out separately).
• Constant column — only one distinct value.
• Likely identifier — every value is unique.
• Surrounding whitespace — values with leading/trailing spaces.

Per dataset:

• Duplicate rows and duplicate column names.

Development

python -m venv .venv
.venv\Scripts\activate            # Windows
# source .venv/bin/activate       # macOS/Linux
pip install -e ".[dev]"
pytest

The package uses a src/ layout:

src/data_profiler/
  cli.py          # Click entry point (the `profiler` command)
  io.py           # CSV/Parquet loading + format detection
  inference.py    # logical type inference from raw values
  checks.py       # data-quality checks
  profiler.py     # orchestration -> DataProfile
  report.py       # text and JSON rendering

You can also use it as a library:

import pandas as pd
from data_profiler.profiler import profile_dataframe

profile = profile_dataframe(pd.read_csv("data.csv"), path="data.csv")
for column in profile.columns:
    print(column.name, column.inferred_type, column.issues)

Publishing to PyPI

Releases are cut by pushing a version tag. The publish workflow then builds the sdist + wheel and uploads them via PyPI Trusted Publishing (no API token needed once the project is configured on PyPI).

# 1. bump the version in src/data_profiler/__init__.py and update CHANGELOG.md
# 2. tag and push
git tag v0.1.0
git push origin v0.1.0

To build and upload manually instead:

python -m build
twine upload dist/*

Note on the package name: PyPI names must be globally unique. If tabular-profiler is taken, change name in pyproject.toml (the import package data_profiler and the profiler command can stay the same).

License

MIT