Tablint

Dataset Profiler & Debugger for Machine Learning

Overview

Tablint is a Python library for intelligent dataset profiling and debugging that acts as a comprehensive pre-training quality assurance tool for machine learning projects. Think of it as "Pandas Profiling + PyLint for datasets", designed specifically for machine learning workflows.

It catches critical dataset issues before they derail your ML pipeline, explains the problems, and suggests context-aware fixes.
If you want, Tablint can even apply those fixes for you automatically.

Features

Key features include:

• Intelligent Profiling: Detect missing values, skewed distributions, outliers, and data type inconsistencies.
• ML-Specific Checks: Identify data leakage, dataset drift, class imbalance, and high-cardinality features.
• Automated Preparation: Get suggestions for encoding, imputation, scaling, and transformations.
• Rich Reporting: Generate statistical summaries and exportable reports (HTML/PDF/Markdown/JSON) with embedded visualizations.
• Reviewed preparation code: Write a pandas cleaning script and a scikit-learn pipeline. Fit the pipeline on the training split. The pandas script computes statistics from the frame you pass in.
• Modern Themes: Choose between "Minimal" (professional) and "Neubrutalism" (bold) report styles.

Installation

Using pip

pip install tablint

Using uv (recommended)

# Install uv if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install tablint
uv pip install tablint

# Or for development from source
git clone https://github.com/cachevector/tablint.git
cd tablint
uv sync

After installation, the tablint command will be available directly in your terminal.

Usage

Tablint can be used both as a command-line tool and as a Python library.

CLI Usage

1. Quick Scan

Get a quick summary of critical issues in your terminal.

tablint scan dataset.csv

Options:

• --critical-only: Show only critical issues
• --quiet: Minimal output (counts only)
• --json: Output in JSON format
• --target COLUMN: Specify target column for ML-specific checks
• --checks CHECKS: Run specific checks (comma-separated)
• --comparison FILE: Compare with another dataset for drift detection
• --preset {gate,profile}: gate is the default and is the training-blocker set. profile runs every check.
• --fail-on {critical,warning}: Exit 2 when that severity is present. Exit 0 means the bar was not hit. Exit 1 is a usage or runtime error.
• --sample-size N: Analyze at most N rows. Sampling stays off unless this is set.
• --no-sample: Disable sampling even when --sample-size is set.
• --config FILE: Load thresholds from a YAML/TOML/JSON config file

Example:

# Block a training job when the table is not trainable
tablint scan train.csv --target Survived --preset gate --fail-on critical

# Quick scan with JSON output
tablint scan dataset.csv --json --quiet

2. Detailed Analysis

Get comprehensive details about all detected issues.

tablint details dataset.csv

Options: Same as scan command (including --config)

Example:

tablint details train.csv --target Survived

3. Generate Reports

Generate comprehensive reports in multiple formats with visualizations.

tablint report dataset.csv --format html --theme minimal

Options:

• --output PATH, -o PATH: Custom output file path
• --format {md,json,html,pdf}: Report format (default: md)
• --theme {minimal,neubrutalism}: HTML report theme (default: minimal)
• --with-code: Generate Python scripts for fixes and pipelines
• --full / --no-full: Include/exclude full summaries (default: True)
• --visualizations / --no-visualizations: Include/exclude plots (default: True)
• --target COLUMN: Specify target column
• --checks CHECKS: Run specific checks
• --comparison FILE: Compare with another dataset for drift detection
• --preset {gate,profile}: profile is the default for reports. gate runs only training blockers.
• --sample-size N: Analyze at most N rows. Sampling stays off unless this is set.
• --no-sample: Disable sampling even when --sample-size is set.
• --config FILE: Load thresholds from a YAML/TOML/JSON config file

Examples:

# Generate HTML report with minimal theme
tablint report dataset.csv --format html --theme minimal --full

# Generate PDF report without visualizations (faster)
tablint report dataset.csv --format pdf --no-visualizations

# Generate report with automatic fix scripts
tablint report dataset.csv --with-code

# Generate report with custom output path
tablint report dataset.csv --format html --output my_reports/analysis.html

# This creates:
# - dataset_tablint_report.md (or .html/.pdf/.json)
# - dataset_tablint_report_fixes.py (pandas script)
# - dataset_tablint_report_pipeline.py (sklearn pipeline)

# Compare two datasets for drift detection
tablint report train.csv --comparison test.csv --format html

4. List Available Checks

Discover all data quality checks that Tablint can perform.

tablint checks

5. Version

Check Tablint version.

tablint version

Available Checks

• outliers - Detect outliers using z-score
• duplicates - Find duplicate rows
• high_missing_values - Columns with high missing data
• empty_columns - Completely empty columns
• dataset_missingness - Overall missing data patterns
• missing_patterns - Correlated missing value patterns
• high_cardinality - Categorical columns with too many unique values
• single_value_columns - Constant columns with no variance
• mixed_data_types - Columns with mixed data types
• class_imbalance - Imbalanced target variable (requires --target)
• feature_correlation - Highly correlated numeric features
• categorical_correlation - Highly associated categorical features
• mixed_correlation - Numeric-categorical associations
• data_leakage - Columns identical to target
• target_leakage_patterns - Features that may leak target information
• dataset_drift - Distribution drift between datasets (requires --comparison)
• uniform_distribution - Uniformly distributed numeric columns
• unique_values - Columns where >95% values are unique
• high_zero_counts - Columns with excessive zero values
• skewness - Highly skewed numeric distributions
• infinite_values - Columns containing infinite values
• constant_length - String columns with constant character length
• extreme_text_lengths - Text columns with extreme value lengths
• datetime_skew - Datetime columns concentrated in one period
• datetime_future_dates - Datetime columns with values in the future
• datetime_gaps - Anomalous gaps in datetime sequences
• datetime_monotonicity - Non-monotonic datetime columns
• normality - Non-normal numeric distributions (Shapiro-Wilk / D'Agostino-Pearson)
• variance_homogeneity - Unequal variances across target groups (Levene's test, requires --target)
• low_mutual_information - Features with near-zero mutual information with the target (requires --target)
• empty_dataset - Empty or all-missing datasets

Python Library Usage

Basic Analysis

import pandas as pd
from tablint import DatasetAnalyzer

# Load your dataset
df = pd.read_csv("dataset.csv")

# Create analyzer
analyzer = DatasetAnalyzer(df)

# Run analysis
summary = analyzer.analyze()

# Access results
print(f"Critical issues: {summary['critical_count']}")
print(f"Warnings: {summary['warning_count']}")

# Iterate through issues
for issue in summary['issues']:
    print(f"{issue['severity']}: {issue['description']}")

Analysis with Target Column

# Specify target for ML-specific checks
analyzer = DatasetAnalyzer(
    df,
    target_col='target_column'
)
summary = analyzer.analyze()

Run Specific Checks

# Only run specific checks
analyzer = DatasetAnalyzer(
    df,
    selected_checks=['outliers', 'high_missing_values', 'class_imbalance']
)
summary = analyzer.analyze()

Include Visualizations

# Generate analysis with plots
analyzer = DatasetAnalyzer(df, include_plots=True)
summary = analyzer.analyze()

# Plots are stored in summary['summaries']['plots']

Drift Detection

# Compare two datasets
train_df = pd.read_csv("train.csv")
test_df = pd.read_csv("test.csv")

analyzer = DatasetAnalyzer(
    train_df,
    comparison_df=test_df,
    selected_checks=['dataset_drift']
)
summary = analyzer.analyze()

Generate Reports Programmatically

from tablint.reports import generate_report

# Analyze dataset
analyzer = DatasetAnalyzer(df, include_plots=True)
summary = analyzer.analyze()

# Generate HTML report
generate_report(
    summary,
    format='html',
    full=True,
    output_file='report.html',
    theme='minimal'
)

# Generate PDF report
generate_report(
    summary,
    format='pdf',
    full=True,
    output_file='report.pdf'
)

# Generate JSON report
generate_report(
    summary,
    format='json',
    full=True,
    output_file='report.json'
)

# Generate Markdown report
generate_report(
    summary,
    format='md',
    full=True,
    output_file='report.md'
)

Generate Fix Scripts

from tablint.checks.core import Issue
from tablint.preparers.codegen import CodeGenerator
from tablint.preparers.pipeline_builder import PipelineBuilder
from tablint.preparers.suggestions import SuggestionProvider

# After running analysis
analyzer = DatasetAnalyzer(df, target_col='target')
summary = analyzer.analyze()

# Convert issues to proper format
issues = [Issue(**i) for i in summary['issues']]
column_types = summary.get('column_types', {})

# Get suggestions
provider = SuggestionProvider(
    issues=issues,
    column_types=column_types,
    target_col='target'
)
suggestions = provider.get_suggestions()

# Generate pandas fix script
codegen = CodeGenerator(suggestions)
fixes_code = codegen.generate_pandas_script()
with open('fixes.py', 'w') as f:
    f.write(fixes_code)

# Generate sklearn pipeline
builder = PipelineBuilder(suggestions)
pipeline_code = builder.generate_pipeline_code()
with open('pipeline.py', 'w') as f:
    f.write(pipeline_code)

Load Config from File

from tablint.utils.config_loader import load_config
from tablint import DatasetAnalyzer

# Load thresholds from YAML, TOML, or JSON
config = load_config("tablint.yaml")  # or .toml / .json

analyzer = DatasetAnalyzer(df, config=config)
summary = analyzer.analyze()

Example tablint.yaml:

missing_values:
  warning: 0.3
  critical: 0.6
outliers:
  z_score: 3.5
statistical_tests:
  normality_p_value: 0.01

Only the keys you specify are overridden; all others fall back to defaults.

Custom Sampling

from tablint.utils.sampling import SamplingConfig

# Configure sampling for large datasets
sampling_config = SamplingConfig(max_rows=10000)

analyzer = DatasetAnalyzer(
    df,
    sampling_config=sampling_config,
    auto_sample=True
)
summary = analyzer.analyze()

# Check if sampling occurred
if 'sampling_info' in summary:
    info = summary['sampling_info']
    print(f"Sampled: {info['sample_fraction']*100:.1f}%")

Production gate

DatasetAnalyzer defaults to the gate preset. tablint report defaults to profile.

critical is reserved for defects that should stop a training job: an empty table, an all-null column, missingness at or above the critical threshold, infinite values, duplicate rows above the critical ratio, a column identical or nearly identical to the target, and drift whose effect size is large. Skew, non-normality, ordinary correlation with the target, and zero-inflation of a binary target are warnings, and most of them run only in the profile preset.

Stable keys in analyze() and in scan --json / details --json:

• schema_version ("0.2"), critical_count, warning_count, total_issues
• issues[]: category, severity, column, description, impact_score, quick_fix, statistic, effect_size
• sampling_info

summaries and plots are best-effort and can change without a major version.

tablint scan train.csv --target Survived --preset gate --fail-on critical --json

Exit 2 when --fail-on matches. A job that only checks the exit code should pass --fail-on critical.

Limitations

• Input is CSV, or Parquet when tablint[parquet] is installed. HTML, PDF, and plots need tablint[reports].
• Leakage means near-deterministic association with the target, not a proof that a column was built from the label.
• Sampling is off unless you pass --sample-size or auto_sample=True. Checks then describe the analyzed frame, which is the full table by default.
• Generated pandas code learns fill values from the frame you pass. Pass the training split, or fit the generated sklearn pipeline on that split.

License

This project is licensed under the MIT License.

Contributing

We welcome contributions from the community to make Tablint better!

Before you get started, please:

• Review our CONTRIBUTING.md for detailed guidelines and setup instructions
• Write clean, well-documented code
• Follow best practices for the stack or component you’re working on
• Open a pull request (PR) with a clear description of your changes and motivation