scomp-link: The Astromech Arm for Your Python Projects

May the code be with you

Overview

scomp-link is an end-to-end machine learning toolkit that automates the complete ML workflow — from data profiling and preprocessing to model selection, training, validation, explainability, monitoring, and deployment.

It includes a full-featured CLI for zero-code ML workflows and a Python API for programmatic use.

Installation

pip install scomp-link

Requires Python 3.10+. Import is near-instant (~6ms) thanks to lazy loading — heavy dependencies load only when needed. Python 3.14 is supported experimentally (TensorFlow not yet available on 3.14).

Key Features

CLI Quick Start

# Scaffold a new project
scomp-link init my_project

# Quick dataset profiling
scomp-link describe --data data.csv --format table

# Full data quality report
scomp-link quality --data data.csv --output report.html

# Feature engineering
scomp-link engineer --data data.csv --target y --interactions --log-transform --output features.csv

# Train a model
scomp-link run --data features.csv --target y --task regression --save-artifact model.scomp

# Train with text data
scomp-link text --data tickets.csv --text-col message --target category --method contrastive --head auto

# Extract embeddings from trained model
scomp-link embed --data new_texts.csv --text-col message --artifact model.scomp --output embeddings.npy

# Clustering
scomp-link cluster --data customers.csv --n-clusters 5 --plot clusters.html

# Hyperparameter tuning
scomp-link tune --data train.csv --target y --task regression --method optuna --n-trials 100

# Predict
scomp-link predict --artifact model.scomp --data new_data.csv --output predictions.csv

# Validate on test data
scomp-link validate --artifact model.scomp --data test.csv --target y --report report.html

# Explain
scomp-link explain --artifact model.scomp --data test.csv

# Detect drift
scomp-link drift --reference train.csv --current production.csv --plot drift.html

# Production monitoring (drift + quality + performance)
scomp-link monitor --reference train.csv --current prod.csv --artifact model.scomp --target y

# Forecast time series
scomp-link forecast --data series.csv --column value --horizon 30 --plot forecast.html

# Anomaly detection
scomp-link anomaly --data data.csv --methods iforest,lof,tabnet,transformer

# Fairness check
scomp-link fairness --data preds.csv --target y_true --predicted y_pred --sensitive gender

# Compare models
scomp-link compare --artifacts v1.scomp v2.scomp --plot comparison.html

# Run full pipeline from YAML config
scomp-link pipeline --config pipeline.yaml

# Serve model as REST API
scomp-link serve --artifact model.scomp --port 8080

# Export model to standard format
scomp-link export --artifact model.scomp --format onnx

# Generate reports
scomp-link report --data data.csv --output eda_report.html
scomp-link report --artifact model.scomp --data test.csv --output model_report.html

# Utilities
scomp-link list-models
scomp-link check-deps

# Configuration
scomp-link init-config              # Create global config (~/.scomp-link/config.yaml)
scomp-link init-config --local      # Create project-level config (.scomp-link.yaml)

Python API Quick Start

from scomp_link import ScompLinkPipeline, ScompArtifact, set_verbosity
import pandas as pd

# Control output
set_verbosity("info")  # "silent" | "warning" | "info" | "debug"

# Build pipeline
pipe = ScompLinkPipeline("My Project")
pipe.set_objectives(["Minimize RMSE"])
pipe.import_and_clean_data(df)
pipe.select_variables(target_col='target')
pipe.choose_model("numerical_prediction")
results = pipe.run_pipeline(task_type="regression")

# Save as artifact
artifact = ScompArtifact()
artifact.set_model(pipe.model)
artifact.set_config(task_type='regression', target_col='target')
artifact.set_metrics(results['metrics'])
artifact.save('model.scomp')

# Load and predict
loaded = ScompArtifact.load('model.scomp')
predictions = loaded.predict(new_data)

Pipeline DSL (>> operator)

Compose ML pipelines and HTML reports declaratively with Airflow-style >> syntax:

from scomp_link import CleanStep, SelectStep, ModelStep, TrainStep

# Lazy pipeline — >> builds the chain, .run() executes it
results = (
    CleanStep(df)
    >> SelectStep("price", features=["sqm", "rooms"])
    >> ModelStep("numerical_prediction")
    >> TrainStep("regression", test_size=0.2)
).run()

Build HTML reports the same way:

from scomp_link import SectionStep, TitleStep, TableStep, GraphStep, SaveStep
from scomp_link.utils.report_html import ScompLinkHTMLReport

report = ScompLinkHTMLReport("Q4 Report")

(
    SectionStep("Results")
    >> TitleStep("Model Performance")
    >> TableStep(metrics_df, "Metrics")
    >> GraphStep(fig, "RMSE Chart")
    >> SaveStep("q4_report.html")
).run(report)

Use LogStep to inspect intermediate state:

from scomp_link import LogStep

(CleanStep(df) >> LogStep("after clean") >> SelectStep("y") >> TrainStep("regression")).run()

Feature Engineering

from scomp_link import FeatureEngineer

fe = FeatureEngineer(
    interactions=True,      # Polynomial interactions
    log_transform=True,     # Log1p for skewed features
    date_features=True,     # Extract year/month/dow/weekend
    target_encode=True,     # Encode high-cardinality categoricals
    auto_bin=True,          # Quantile binning
)
X_train_eng = fe.fit_transform(X_train, y_train)
X_test_eng = fe.transform(X_test)

Advanced Hyperparameter Tuning

from scomp_link.models.advanced_tuning import OptunaOptimizer

def param_space(trial):
    return {
        'n_estimators': trial.suggest_int('n_estimators', 50, 500),
        'max_depth': trial.suggest_int('max_depth', 3, 20),
        'learning_rate': trial.suggest_float('learning_rate', 0.01, 0.3, log=True),
    }

optimizer = OptunaOptimizer(GradientBoostingRegressor, param_space, scoring='r2', n_trials=100)
best_model = optimizer.optimize(X_train, y_train)

Explainability

from scomp_link import ShapExplainer, LimeExplainer

# SHAP
shap_exp = ShapExplainer(model, X_train[:100])
shap_exp.explain(X_test)
importance = shap_exp.feature_importance()
fig = shap_exp.plot_importance()

# LIME
lime_exp = LimeExplainer(model, X_train, task='regression')
exp = lime_exp.explain_instance(X_test.iloc[0])
fig = lime_exp.plot_explanation(exp)

Data Drift Detection

from scomp_link import DriftDetector

detector = DriftDetector(X_train, psi_threshold=0.2)
report = detector.detect(X_production)
summary = detector.summary(report)
fig = detector.plot_drift_report(report)

Fairness & Bias Metrics

from scomp_link import FairnessMetrics

fm = FairnessMetrics(y_true, y_pred, sensitive_feature=df['gender'])
report = fm.compute_all()
print(fm.summary(report))
fig = fm.plot_fairness_report(report)

Time Series Forecasting

from scomp_link import TimeSeriesForecaster

fc = TimeSeriesForecaster(method='auto', horizon=30)
fc.fit(series)
forecast = fc.predict_with_ci()
cv_results = fc.walk_forward_cv(series, n_splits=5)
fig = fc.plot_forecast()

Data Quality Report

from scomp_link import DataQualityReport

dqr = DataQualityReport(df)
report = dqr.generate()  # missing, cardinality, constants, duplicates, correlations
dqr.save_html('quality_report.html')

Anomaly Detection

from scomp_link import AnomalyDetector

detector = AnomalyDetector(
    contamination=0.05,
    methods=['iforest', 'lof', 'tabnet', 'transformer'],
    consensus_threshold=2,
)
results = detector.fit_predict(df, features=['col1', 'col2', 'col3'])

Project Structure

scomp_link/
├── cli.py                    # CLI (24 commands)
├── core.py                   # ScompLinkPipeline orchestrator
├── preprocessing/
│   ├── data_processor.py     # Preprocessor (polars backend)
│   ├── feature_engineer.py   # FeatureEngineer (sklearn-compatible)
│   └── data_quality.py       # DataQualityReport
├── models/
│   ├── model_factory.py      # Decision-tree model selection
│   ├── regressor_optimizer.py
│   ├── classifier_optimizer.py
│   ├── ensemble_optimizer.py
│   ├── advanced_tuning.py    # Optuna, Halving, EarlyStopping
│   ├── forecaster.py         # TimeSeriesForecaster
│   ├── anomaly_detector.py
│   ├── ts_anomaly_detector.py
│   ├── contrastive_text.py   # BERT contrastive learning
│   ├── supervised_text.py
│   └── supervised_img.py
├── validation/
│   ├── model_validator.py    # Metrics + HTML reports
│   ├── advanced_cv.py        # LOOCV, Bootstrap
│   └── fairness.py           # FairnessMetrics
├── explainability/
│   └── explainer.py          # ShapExplainer, LimeExplainer
├── monitoring/
│   └── drift_detector.py     # DriftDetector (PSI + KS)
├── persistence/
│   └── artifact.py           # ScompArtifact (.scomp format)
└── utils/
    ├── colors.py             # Centralized color palettes
    ├── logger.py             # Configurable logging
    ├── report_html.py        # HTML report builder
    ├── plotly_utils.py       # Plotly chart utilities
    ├── highcharts.py         # Highcharts visualizations
    └── rawgraphs/            # 31 SVG chart functions (server-side)

AI Agent Integration

scomp-link works natively with AI agents via MCP (Model Context Protocol) and Agent Skills.

MCP Server (22 tools for structured agent calls)

pip install scomp-link[mcp]
scomp-link mcp  # Starts MCP server (stdio transport)

Claude Desktop (claude_desktop_config.json):

{"mcpServers": {"scomp-link": {"command": "scomp-link", "args": ["mcp"]}}}

Cursor (plugin — one click):

Kiro (.kiro/mcp.json):

{"mcpServers": {"scomp-link": {"command": "scomp-link", "args": ["mcp"]}}}

Remote (no install needed) — connect to the hosted MCP server on 🤗 Hugging Face Space or 🔧 Smithery:

{"mcpServers": {"scomp-link": {"url": "https://Euribor512-scomp-link.hf.space/sse"}}}

Docker:

docker pull jack15121/scomp-link:latest
docker run -i jack15121/scomp-link mcp

Available tools: describe_data, train_model, predict, validate_model, detect_drift, detect_anomalies, check_fairness, forecast_series, engineer_features, cluster_data, generate_report, create_visualization, compare_models, export_model, embed_text, select_backbone, report_create, report_add_section, report_add_text, report_add_table, report_add_chart, report_save

Quick Setup Prompt for AI Agents

Copy-paste this prompt into your AI agent (Claude, ChatGPT, Cursor, etc.) to enable scomp-link capabilities:

You have access to scomp-link, an ML toolkit with 22 MCP tools. Use them for:

• Data profiling: describe_data(path) — always start here
• Training: train_model(data, target, task) with optional tune=true for Optuna
• Validation: validate_model(artifact, data, target) for test evaluation
• Reports: Use the report builder for custom dashboards:
  1. report_create(title) → get report_id
  2. report_add_section(id, title) → structure
  3. report_add_chart(id, engine, type, data, title) → 39 chart types (plotly/rawgraphs/highcharts)
  4. report_add_table(id, json_data, title) → data tables
  5. report_save(id, path) → save HTML
• Monitoring: detect_drift, detect_anomalies, check_fairness
• Forecasting: forecast_series(data, column, horizon)

Report branding defaults come from ~/.scomp-link/config.yaml. Run scomp-link init-config to set up.

Agent Skill (zero-dependency documentation)

# For Kiro
cp -r skills/scomp-link ~/.kiro/skills/

# For Claude Code
cp -r skills/scomp-link .claude/skills/

See AGENT_INTEGRATION.md for full setup guide.

Report Builder (MCP)

Build fully custom branded HTML reports step-by-step via MCP tools:

# 1. Configure corporate defaults (one-time setup)
# scomp-link init-config
# Edit ~/.scomp-link/config.yaml with your branding

# 2. Create a report (uses config defaults automatically)
report_create("Q4 Performance Report")

# 3. Add content
report_add_section(report_id, "Executive Summary")
report_add_table(report_id, metrics_json, "Key Metrics")
report_add_chart(report_id, "plotly", "linechart", data, "Revenue Trend")
report_add_chart(report_id, "rawgraphs", "treemap", data, "Market Share")

# 4. Save
report_save(report_id, "q4_report.html")

Configuration precedence: .scomp-link.yaml (local) > ~/.scomp-link/config.yaml (global) > built-in defaults

# Create global config with your corporate branding
scomp-link init-config

# Or create project-level config
scomp-link init-config --local

Testing

# Run all tests
pytest tests/ -v

# With coverage
pytest tests/ --cov=scomp_link --cov-report=html

Documentation

Full documentation with API reference and CLI guide:

pip install mkdocs mkdocs-material "mkdocstrings[python]"
mkdocs serve  # http://localhost:8000

Contributing

git clone https://github.com/GiacomoSaccaggi/scomp-link.git
cd scomp_link
pip install -e ".[dev]"
pytest tests/ -v

License

MIT License

May the code be with you. 🚀

📦 scomp-link on PyPI