tom-analytics 1.2.0

A one-line autonomous exploratory data analysis (EDA) and reporting library.

🚀 tom-analytics — One-Line Autonomous Data Analytics & EDA Library

tom (published as tom-analytics) is a zero-configuration, single-line automated exploratory data analysis (EDA) and diagnostics engine. Designed for developers, data scientists, and analysts, it handles everything from automatic file loading to sophisticated statistical tests, interactive Plotly visualizations, and premium glassmorphism HTML reports—all from a single function call.

---

🗺️ How it Works (Under the Hood)

The library is designed to go from raw file to interactive, publication-ready report in one line. Here is the operational workflow:

graph TD
    A[Raw File / DataFrame] -->|mos.file| B(Parser & Format Detector)
    B -->|Automatic Type Inference| C(Cleaner & Imputer)
    C -->|Auto-Impute / Drop Dups| D(Autonomous Stats Engine)
    D -->|Parametric & Non-Parametric Tests| E(Visual Suite Generator)
    E -->|Seaborn & Plotly Engines| F(Glassmorphic HTML Compiler)
    F -->|Self-Contained Report| G[report.html & plotly_dashboard.html]

---

✨ Features & Capabilities

• 📂 Universal File Parser (mos.file): Delimiter-independent parser supporting CSV, TSV, JSON, Excel, and Parquet. Features interactive path prompt and string-matching path suggestions.
• 🧹 Autonomous Preprocessing & ID Isolation: Intelligent type coercion (dates, numerical objects), median/mode missing value imputation, automatic column pruning (drops column with $>50%$ missing values), and duplicate row filtering.
  • Robust ID Isolation: Automatically identifies database identifiers, sequential row indices, and UUIDs (e.g. roll_id, user_id). Preserves them in the active DataFrame to prevent downstream pipeline crashes, but completely isolates them from correlation heatmaps, ANOVA tests, and statistical summaries.
• 📊 Multi-faceted Statistical Engine: Computes high-fidelity parametric/non-parametric metrics, Chi-Square tests for category associations, and ANOVA tests for continuous-categorical relationships.
• 🎨 Vibrant Visualizations Suite: High-res static plots (ECDF, Q-Q, violin, box plots, histograms, pairplots, missing value heatmaps) at 300 DPI alongside a fully interactive pan/zoom Plotly dashboard.
• 💡 Context-Aware Analytics & NLP Insights: Generates 10–20 plain-English mathematical diagnostics that avoid common automated EDA interpretation pitfalls:
  • Math-Safe Transformation Suggestion: Checks numerical column bounds before suggesting log/Box-Cox transformations, ensuring log1p or Yeo-Johnson are recommended if 0 or negative values are present to prevent code crashes.
  • Heavy-Tails vs. Outliers: Automatically identifies power-law or log-normal distributions. Warns against blindly dropping outliers that are legitimate heavy-tail values, recommending robust scaling (RobustScaler) instead.
  • Actionable Class Imbalance Advice: Highlights dominant categorical frequencies ($>70%$) and suggests precise code tactics, such as stratified splitting (stratify=y), class weighting (class_weight='balanced'), and tracking F1-score/Precision-Recall rather than overall accuracy.
  • Subgroup Normality Checks: Conducts Shapiro-Wilk testing at the subgroup level of categorical features. If a distribution is non-normal overall but normal within subgroups, it notes that parametric tests (ANOVA, t-tests) remain statistically valid.
• 🪟 Glassmorphic HTML Compiler: Renders a premium, self-contained, completely portable HTML dashboard (./tom_report/report.html) utilizing elegant card spacing, backdrop blurs, dark purple-to-blue gradients, and collapsible diagnostic panels.
• 🖥️ Windows CP1252 Fixes Out-of-the-Box: Process-level sys.stdout and sys.stderr are automatically reconfigured to UTF-8 to prevent any Windows encoding crashes when displaying terminal emojis.

---

🛠️ Installation

You can install the package directly from PyPI or locally in editable mode.

Option A: Install from PyPI (Recommended)

pip install tom-analytics

Option B: Local Developer Installation (Editable Mode)

1. Clone your repository:

   git clone https://github.com/OmkarSugave/tom.git
   cd tom
   

2. Activate your virtual environment and install:

   # Windows PowerShell
   .\venv\Scripts\Activate.ps1
   pip install -e .
   

---

🚀 Quickstart & Essential API Reference

Import tom as mos and unleash automated EDA immediately:

import tom as mos

# 1. Load your dataset (automatically handles format & paths)
mos.file("mock_data.csv")

# 2. Run the complete autonomous pipeline!
mos.describe()

---

📖 Deep API Reference

1. File Loading & Parse: mos.file(path=None)

Loads any tabular dataset. If no path is provided, it opens an interactive terminal dialogue.

# Direct path loading
mos.file("dataset.parquet")

# Delimiter, type, and encoding detection are completely autonomous.
# If you make a typo, tom will suggest the nearest file name!

2. Auto-Cleaning: mos.clean_report(df=None)

Runs and prints a detailed analysis of missing data, data types, duplicate rows, and imputed actions.

mos.clean_report()

3. Outlier Analysis: mos.outliers(df=None)

Runs an IQR-based boundary analysis across all numerical features, displaying lower/upper limits, min/max outliers, and percentages.

mos.outliers()

4. Association & Correlation: mos.correlation(df=None)

Calculates the Pearson correlation matrix for numerical features. Saves a publication-quality heatmap at ./tom_report/charts/single_correlation_heatmap.png and prints a formatted terminal table.

mos.correlation()

5. Automated Machine Learning Guide: mos.suggest(df=None)

Analyzes dataset dimensions, distribution, and types to recommend a classification or regression pipeline. Suggests the best-suited machine learning models (e.g., Random Forests, XGBoost, Ridge) and lists essential preprocessing rules.

mos.suggest()

6. Side-by-Side Comparison: mos.compare(df2, df1=None)

Compares two DataFrames (e.g., training vs. test, or original vs. cleaned) showing shapes, missing values, duplicates, and column overlaps.

import pandas as pd
df_test = pd.read_csv("test_dataset.csv")

mos.compare(df_test)

7. Exporting Reports: mos.export(format_type="pdf")

Exports your gorgeous glassmorphic HTML report into a standard PDF page.

mos.export("pdf")

Note: PDF export utilizes pdfkit (requires wkhtmltopdf on your system path).

---

📁 Package Architecture & Directory Structures

The codebase is highly modularized, clean, and extensible:

tom/
├── tom/
│   ├── __init__.py      # Global UTF-8 reconfiguration, state manager, and public API
│   ├── loader.py        # Autonomous format parses (CSV, TSV, Parquet, JSON, Excel)
│   ├── cleaner.py       # Imputers, duplicate drops, type inference engine
│   ├── stats.py         # Shapiro-Wilk, ANOVA, Chi-Square association engine
│   ├── charts.py        # 300 DPI Seaborn plots + base64 image compiler
│   ├── insights.py      # NLP-driven analytics alerts & warning rules
│   ├── reporter.py      # Glassmorphic HTML template builder & Plotly compiler
│   └── utils.py         # Directory checkers, path string metrics
├── pyproject.toml       # Build metadata & dependency declarations
├── requirements.txt     # Standard pip freeze
├── README.md            # Comprehensive documentation (this file)
└── tom_report/          # Generated output folder (auto-excluded in .gitignore)
    ├── report.html      # Premium portable HTML report
    └── charts/
        └── plotly_dashboard.html  # Interactive pan & zoom dashboard

---

🔒 License

This project is licensed under the MIT License - see the LICENSE file for details.