CUTLASS

CUTLASS (Critical-range rectified LASSO) packages the workflow developed in the project scripts into a reusable, publishable Python library. It exposes a scikit-learn inspired estimator that rectifies the input space into {-1, +1} indicators, trains an L1-penalised logistic model with an efficient coordinate-descent solver, and optionally compresses the model into a logical rule without any dependence on scikit-learn itself. Version 0.8.0 adds ordered FP64 CUDA coordinate descent on top of the persistent multi-fit executor while preserving NumPy coordinate descent as the default and scientific reference.

This project is a statistical modelling package and is not NVIDIA's C++ CUTLASS linear-algebra library.

Features

• Rectifier transformer that infers critical ranges from the positive class and binarises features into {-1, +1}.
• Cross-validated L1 logistic model with warm-started coordinate descent and optional FISTA solver.
• Optional CUDA execution through CuPy for parity-oriented coordinate descent, FISTA, and hybrid GPU-FISTA/CPU-coordinate-descent fitting.
• Adaptive-L1 mode (penalty="adaptive_l1") that fits an L2 logistic pilot, reweights the L1 penalty by abs(beta_pilot) + adaptive_eps, and maps coefficients back to the original feature scale.
• Logical compression step mirroring the research code (top-k votes with fixed magnitude K and several intercept policies).
• Serialization helpers to persist rectifier limits, fitted weights, and backend provenance.
• Observable execution through backend reports, synchronized phase timings, progress callbacks, cancellation, and GPU memory/transfer diagnostics.
• Persistent multi-fit CUDA execution with bounded explicit streams, input-ordered results, incremental callbacks, aggregate Auto selection, resident input caching, memory admission, and visible fallback policies.
• Lightweight CPU installation based on NumPy and pandas. Matplotlib and CuPy are optional extras for plots and CUDA execution respectively.

Execution model

CPU remains the default so existing results and installations are unchanged. The backend argument is available on both CutlassLogisticCV and CutlassClassifier:

Solver support is explicit:

Adaptive L1 is supported by cd, fista, and hybrid. Logical polishing is always a CPU post-processing phase, including after a CUDA fit.

Installation

pip install cutlass

The plotting utilities used by the logical compression step are optional. To enable them, install the plots extra:

pip install cutlass[plots]

CUDA is optional and requires a compatible NVIDIA driver. Install exactly one CuPy provider matching the CUDA major version supported by the environment:

pip install "cutlass[cuda13]"

Use cuda12 instead for a CUDA 12 environment. Do not install multiple CuPy distributions in the same environment. CuPy is imported lazily, so the base package remains usable on systems without CUDA.

Quick start

import pandas as pd
from cutlass import CutlassClassifier

# toy binary dataset
df = pd.DataFrame(
    {
        "feat_a": [0.1, 0.3, 0.7, 0.9, 0.2, 0.8],
        "feat_b": [10, 13, 8, 5, 11, 4],
        "INDC": [0, 0, 1, 1, 0, 1],
    }
)

X = df.drop(columns=["INDC"])
y = df["INDC"]

clf = CutlassClassifier(
    rectify=True,
    Cs=15,
    solver="cd",
    cv=3,
    logic_polish=True,
    logic_scale=10.0,
)
clf.fit(X, y)
print(clf.predict_proba(X))
print("limits:", clf.limits_)

The default penalty remains standard L1. To use the adaptive-L1 mode, pass the optional penalty argument:

adaptive_clf = CutlassClassifier(
    rectify=True,
    Cs=15,
    solver="cd",
    cv=3,
    penalty="adaptive_l1",
    adaptive_eps=1e-3,
)
adaptive_clf.fit(X, y)

To reproduce the canonical coordinate-descent algorithm on CUDA throughout CV and the final refit:

from cutlass import CutlassLogisticCV, probe_backend

print(probe_backend("cuda", device=0).to_dict())

gpu_model = CutlassLogisticCV(
    Cs=15,
    cv=3,
    solver="cd",
    backend="cuda",
    device=0,
    dtype="float64",
    allow_cpu_fallback=True,
)
gpu_model.fit(X.to_numpy(), y)
print(gpu_model.backend_used_)
print(gpu_model.backend_report_)

solver="cd" performs strong-rule screening, ordered coordinate updates, KKT checks, warm starts, CV, and the final refit on CUDA. Its report identifies implementation="cuda_cd_v1" and parity_profile="cpu_cd_fp64_v1". solver="fista" runs both CV and final fitting on CUDA. solver="hybrid" retains its distinct FISTA-CV/CPU-CD-final-refit contract and should not be used as a substitute when CPU/CD ranking parity is required.

backend="auto" uses a deterministic policy. It currently selects CUDA for a compatible solver when n_rows * n_features * n_folds * n_C_values is at least 75,000,000 work units (doubled for adaptive L1), unless CUTLASS_CUDA_AUTO_MIN_WORK overrides that threshold. This prevents transfer and startup overhead from slowing down small fits.

For solver="cd", that threshold is a shared routing heuristic, not a measured CPU/CD-to-CUDA/CD performance crossover. The ordered CUDA/CD implementation synchronizes after dependent coordinate reductions to preserve the CPU/CD algorithm. In the committed small-fit RTX 3090 matrix it was 27--34 times slower than CPU/CD, and no single-fit crossover has yet been demonstrated. Auto can therefore choose a supported, sufficiently large CUDA/CD workload without promising that it will be faster.

CUDA/CD is most likely to become competitive for very tall matrices with a low or moderate feature count, or for large batches of independent fits whose data remain resident on the device. Small individual fits, wide matrices, and jobs that repeatedly transfer state to the host normally favor CPU/CD. Treat backend="cuda" as a parity choice first and a performance choice only after benchmarking the complete application workload. The validation evidence and benchmark command are recorded in the CUDA/CD implementation guide.

CUDA inputs may be NumPy arrays or CuPy device arrays. Fitted public attributes and predictions are returned as NumPy arrays so serialization and downstream code behave the same on every backend.

Progress, cancellation, and diagnostics

Long-running fits can report phase progress and stop cooperatively:

cancelled = False
gpu_model.fit(
    X.to_numpy(),
    y,
    progress_callback=lambda event: print(
        event["phase"], event["completed"], event["total"]
    ),
    cancel_callback=lambda: cancelled,
)

After fitting, inspect backend_requested_, backend_used_, backend_provider_, device_name_, dtype_, n_jobs_effective_, auto_decision_, fit_timings_, and backend_report_. The report also records fallback reasons, runtime versions, transfers, synchronization points, and peak observed GPU memory. Backend discovery is available through list_devices() and probe_backend() without constructing an estimator.

Persistent multi-fit CUDA execution

Applications with many independent fits can reuse one device context and overlap complete estimator fits on bounded non-default streams:

from cutlass import CudaFitExecutor, CutlassLogisticCV, FitRequest

requests = [
    FitRequest(
        key=f"job-{index}",
        estimator=CutlassLogisticCV(
            Cs=5,
            cv=3,
            solver="cd",
            backend="cuda",
            device=0,
            allow_cpu_fallback=False,
            verbose=False,
        ),
        X=X_train,
        y=y_train,
        metadata={"caller_index": index},
    )
    for index in range(20)
]

with CudaFitExecutor(device=0, max_streams="auto") as executor:
    batch = executor.fit_many(requests)

print([result.status for result in batch.results])
print(batch.diagnostics)

The returned result list always follows input order, even when fits finish out of order. Use CacheIdentity for persistent X/y reuse, result_callback for incremental terminal results, and fallback_policy="none", "defer", or "after_batch" for explicit CPU routing. One executor belongs to one process and one physical GPU; application queues remain application-owned. max_streams="auto" has been deliberately calibrated to one stream since 0.7.0 because the complete-fit scheduler's reference benchmark was faster serially. Values from 2 through 8 remain available for applications that measure a benefit on their own workload.

Vignettes

Additional step-by-step guides live under docs/vignettes/:

• Basic rectified workflow - reproduce the CPU reference fit.
• Logical polish - enable logical compression and interpret diagnostics.
• Batch experiments - run experiments and retain backend provenance.
• GPU backend - configure CUDA, Auto mode, fallback, progress, and persistent services.
• GPU multi-fit - overlap independent fits with a persistent stream executor and resident cache.
• GPU implementation - architecture, delivered scope, and validation status.
• GPU enhancement implementation - multi-fit architecture, delivered scope, and remaining optimization plan.
• GPU coordinate descent - ordered CUDA/CD contract, diagnostics, parity gates, and operating envelope.
• Changelog - release-level capability history.

API highlights

• cutlass.Rectifier: transformer implementing the critical-range binarisation.
• cutlass.CutlassLogisticCV: lower-level L1 or adaptive-L1 logistic with cross-validation.
• cutlass.CutlassClassifier: full workflow composed of the rectifier, optional scaling, and the logistic path solver. Use penalty="l1" for the default behavior or penalty="adaptive_l1" for the adaptive mode.
• cutlass.list_devices and cutlass.probe_backend: runtime discovery and an allocation-based health check for applications and service startup.
• cutlass.FitProgress: the schema used to create JSON-safe progress dictionaries delivered to callbacks.
• cutlass.FitRequest, cutlass.FitResult, and cutlass.FitBatchResult: generic multi-model request and ordered-result contracts.
• cutlass.CudaFitExecutor and cutlass.fit_many: persistent and temporary single-device multi-fit execution.
• cutlass.CacheIdentity and cutlass.BatchFitProgress: safe resident input reuse and JSON-compatible batch progress.
• cutlass.BackendUnavailableError, cutlass.BackendConfigurationError, cutlass.BackendExecutionError, and cutlass.FitCancelledError: actionable execution failures that applications can handle separately.
• cutlass.serialization: helpers for saving rectifier limits and fitted weights. Model artifacts include a JSON-safe backend provenance report.

Refer to the docstrings for detailed parameter descriptions; they mirror the research scripts so existing experiment drivers can be migrated with minimal changes.

Development

To build the package locally:

python -m build

To update the project on PyPI, first bump version in pyproject.toml, commit the release changes, and create a clean source/wheel build with python -m build. After confirming the files under dist/ are correct, upload them with python -m twine upload dist/* using an account or API token that has permission to publish the cutlass package.

Run the CPU suite on any supported Python environment:

python -m pytest -m "not cuda"

In an environment with a usable NVIDIA GPU and CuPy provider, run the complete suite (CUDA tests skip automatically when the runtime is unavailable):

python -m pytest

License

MIT License. See LICENSE for details.