separatix 0.1.0a2

Diagnostic profiling of labeled embeddings for classification model complexity guidance.

separatix

separatix profiles labeled feature spaces before classifier training and returns transparent, confidence-aware guidance about apparent classification complexity.

The intended use case includes learned embeddings, but the package is not restricted to embeddings. It also works on raw feature matrices when you want a coarse diagnostic of whether the observed class geometry looks mostly linear, smoothly nonlinear, local or kernel-like, fragmented, bottlenecked, or too unreliable to trust.

separatix does not claim to pick the optimal classifier. It is a pretraining diagnostic and auditing tool designed to make its reasoning visible.

Installation

pip install separatix

To install the latest development version directly from GitHub:

pip install "git+https://github.com/NiklasMelton/Separatix.git@develop"

Quick start

from separatix import diagnose

recommendation = diagnose(X, y, random_state=0)
print(recommendation)

For a structured audit:

from separatix import diagnose

report = diagnose(X, y, return_report=True, random_state=0)
print(report.recommendation_text)
print(report.decision_path)
print(report.scores)
print(report.to_json())

What It Accepts

• Dense NumPy arrays
• SciPy sparse matrices
• pandas DataFrames and Series when pandas is installed
• Binary and multiclass classification targets
• String or numeric labels treated as categorical class identifiers

Regression, multilabel classification, and multioutput classification are not supported.

What It Returns

By default, diagnose(...) returns a plain-text recommendation. With return_report=True, it returns a DiagnosticReport that includes:

• the recommendation label
• plain-text recommendation text
• confidence level
• underlying metric groups
• probe-family evidence, including uncertainty-aware family comparisons
• normalized summary scores
• a visible decision path
• warnings and skipped diagnostics
• sampling and densification events
• preprocessing and runtime metadata

The report is JSON-serializable through report.to_dict() and report.to_json().

Recommendation Categories

• linear_likely_sufficient
• smooth_nonlinear_recommended
• kernel_or_local_recommended
• high_capacity_or_partitioning_recommended
• feature_or_label_bottleneck_likely
• insufficient_data_or_unreliable_geometry
• inconclusive

These categories are intentionally coarse. They describe the apparent geometry and difficulty of the labeled feature space, not a guaranteed best model choice.

The synthetic recommendation ladder below shows how separatix responds as the designed dataset geometry moves from simple linear structure toward smoother nonlinearity, local or kernel-like structure, fragmented boundaries, and finally weak-signal or random-label bottlenecks. The x-axis is the intended dataset complexity, while the y-axis is the coarse recommendation level reported by separatix.

Decision Pipeline

The recommendation is produced by a fixed, inspectable pipeline:

1. Validate inputs and encode labels.
2. Audit class counts, imbalance, sparsity, and basic dataset conditions.
3. Compute geometry, neighborhood, boundary, fragmentation, and optional topology diagnostics.
4. Run simple probe models and compare them to a dummy baseline.
5. Build probe-family evidence with uncertainty estimates for linear, smooth_nonlinear, and local_kernel.
6. Apply a 95% signal-vs-dummy gate before making any model-family recommendation.
7. Use conservative escalation: keep the simpler family unless a more complex family has a clear uncertainty-adjusted advantage.
8. Render both a plain-language summary and a structured report, including raw_best_family and recommended_family when a report is requested.

The full rationale and decision rules are documented in docs/decision_pipeline.md.

Sparse Inputs And Memory Behavior

Sparse matrices are accepted directly. Diagnostics that need dense data use a shared densification policy rather than a separate dense-only code path. When a step would require densification, separatix can fail, skip, or warn and subsample before densifying, depending on configuration. These events are recorded in the report.

Examples

• examples/basic_breast_cancer.py
• examples/linear_hyperplane_visual.py
• examples/curvilinear_boundary_visual.py
• examples/high_dimensional_linear_hyperplane.py
• examples/high_dimensional_curvilinear_hyperplane.py
• examples/moons_vs_linear.py
• examples/circles_kernel_signal.py
• examples/recommendation_complexity_ladder.py
• examples/multiclass_wine.py
• examples/sparse_text_like_embeddings.py

Related Work

This package is not an implementation of a published dataset-complexity procedure, but the project is adjacent to and inspired by prior work on classification complexity and data geometry. In particular, would like to acknowledge:

• Ho and Basu, "Complexity Measures of Supervised Classification Problems" (PDF)
• Lorena, Garcia, Lehmann, Souto, and Ho, "How Complex Is Your Classification Problem? A Survey on Measuring Classification Complexity" (DOI, PDF)

We do not follow those procedures directly, but they are relevant background for why geometry-aware pretraining diagnostics are useful.