CLI Dataset Inspector

English | Русский

Inspect a tabular dataset, find data quality issues, and generate a self-contained Jupyter Notebook with exploratory analysis, preprocessing and an optional baseline model.

Installation

Requires Python 3.10 or newer. Before the first PyPI release, install from a local checkout:

python -m pip install .

For development:

python -m venv .venv
# Windows PowerShell: .venv\Scripts\Activate.ps1
# macOS/Linux: source .venv/bin/activate
python -m pip install -e ".[dev]"

After publication, installation will be: python -m pip install cli-dataset-inspector. All supported file readers and baseline dependencies are installed with the package. To open notebooks, use a Jupyter-capable editor or install JupyterLab separately.

Quick start

dataset-inspect examples/customers.csv
dataset-inspect examples/customers.csv --target churn
dataset-inspect examples/customers.csv --json
dataset-inspect examples/customers.csv --report-output reports/customers.json
dataset-inspect examples/customers.csv --jupyter --output reports/eda.ipynb
dataset-inspect examples/customers.csv --target churn --baseline --output reports/model.ipynb

python -m dataset_inspector is equivalent to dataset-inspect.

The CLI never trains a model. --baseline implies --jupyter and requires --target. It adds executable preprocessing, training and evaluation cells to the notebook. Open the notebook and run its cells to train the model.

Input formats

CSV/TSV and JSON also support pandas-inferred compression such as .csv.gz and .jsonl.gz. Use --format csv|excel|json|jsonl|parquet to override extension detection.

dataset-inspect data.csv --sep ";" --encoding cp1251
dataset-inspect book.xlsx --sheet "Sales"
dataset-inspect book.xlsx --sheet 1
dataset-inspect book.xlsx --sheet name:0
dataset-inspect records.jsonl --json
dataset-inspect data.parquet --target price --task regression --baseline
dataset-inspect data.txt --format csv

--sheet name:0 selects a sheet literally named 0; --sheet 0 selects the first sheet. Input paths are local files. Nested JSON/Parquet values must be flattened first. Column names are normalized to strings and must remain unique.

Reports and warnings

The console and JSON report include:

• Rows, columns, memory usage in bytes, duplicate count and percentage.
• Per-column dtype, filled/missing counts, missing percentage, unique count and percentage.
• Numeric min, max, mean, median, standard deviation and outlier count using the 1.5 × IQR rule.
• Category counts, most frequent values and the top three values.
• Optional target summary, detected task and class distribution or regression statistics.
• Data quality warnings.

Warnings cover empty datasets/columns, missing values at or above 30%, constant columns, possible IDs (at least 95% unique and more than 20 distinct values), mixed or suspicious numeric/text values, infinite numeric values, duplicate rows, and class imbalance (smallest/largest class count below 0.25).

These are heuristics, not proof of a problem. Numeric summaries exclude infinity. Undefined numeric statistics are null in JSON and n/a in the console. Outliers are reported in numeric summaries, not automatically removed.

JSON output

--json writes one strict JSON document to stdout with no tables or status text. Status/error messages use stderr. --report-output PATH saves the same report as UTF-8 JSON and can be combined with either console or JSON output.

dataset-inspect examples/customers.csv --json --target churn --report-output reports/result.json

The report has schema_version: "1.0" and these top-level keys: shape, duplicates, missing, memory, columns_summary, numeric_summary, categorical_summary, target, warnings. target is null when unspecified. It contains no trained model or metrics; training only happens in the notebook.

Notebook and baseline

Generated notebooks include data loading, general statistics, missing values, duplicates, a generation-time warning snapshot, histograms, category frequencies, correlations, and optional target analysis.

The load cell preserves the format, separator, encoding and sheet settings. The notebook reads the original file through an editable absolute DATASET_PATH; it does not embed the dataset and runs without the dataset_inspector package. It needs pandas, numpy, matplotlib, scikit-learn, IPython/Jupyter and the relevant reader library.

Without --task, nonnumeric/bool targets and numeric targets with at most 20 unique nonmissing values are classified; other numeric targets use regression. Use --task classification or --task regression to override this heuristic consistently in the report and notebook. Regression requires a numeric target.

Baseline behavior:

• Exclude missing/infinite targets; treat infinite numeric features as missing.
• Numeric features: median imputation and standard scaling.
• Categorical features: mode imputation and one-hot encoding that tolerates unseen values.
• Dates and other nonnumeric features are treated as categories; review them for your domain.
• All learned transformations fit on training data only.
• 80/20 train/test split, seed 42; classification uses stratification.
• LogisticRegression: accuracy, macro F1 and a precision/recall/F1 classification report.
• LinearRegression: MAE, RMSE and R².
• If rows/classes/features cannot support a valid split, the notebook skips training and explains why.

Change the split with --test-size 0.3 --random-state 7, or edit the configuration cell. After execution, baseline_result contains status and metrics (or a skip reason), and model is the fitted scikit-learn pipeline when training succeeds. The CLI itself does not execute notebooks or save trained model files.

Plots and correlation matrices are limited to the first 20 relevant columns. Edit MAX_PLOTS in the notebook to change this. Inspection loads the full dataset into memory. Very large datasets and high-cardinality categoricals can be expensive. Random holdout evaluation assumes independent rows: review duplicates, identifiers, time/group structure and leakage before interpreting metrics.

Output paths and errors

• --output PATH / -op PATH: notebook .ipynb file or directory; needs --jupyter or --baseline.
• Default notebook path: <dataset_stem>_analysis.ipynb in the current directory.
• --report-output PATH: exact JSON report file path.
• Existing outputs require --overwrite; an input dataset can never be an output.
• Output parents are created as needed; each file is written atomically.
• There is no transaction across multiple output files: a later write failure can leave an earlier completed artifact in place.
• Exit codes: 0 success, 1 input/read/write failure, 2 invalid CLI options.
• Header-only tables can be inspected. Zero-byte files and tables with no columns are rejected.
• --help and --version / -v work without a dataset path.

Python API

from dataset_inspector.inspector import Inspector
from dataset_inspector.notebook import create_notebook

report = Inspector("data.csv", target="label").inspect()
create_notebook(
    "data.csv", target="label", data=report,
    baseline=True, output="analysis.ipynb",
)

The API returns ordinary JSON-compatible Python values. Reader settings are keyword arguments to Inspector and passed as load_options={...} to create_notebook.

Development and release

python -m pytest
python -m ruff check src tests
python -m ruff format --check src tests
python -m build
python -m twine check dist/*

Tests exercise real input formats, error handling, output protection and generated notebook execution, including a real Jupyter kernel. The CI configuration tests Python 3.10–3.14 on Linux and Python 3.14 on Windows, and builds/checks the distributions. See the release checklist before publishing.

MIT license.