gradtracer 0.7.5

Flow-based Diagnostics for Embedding Systems & Compression

🌊 GradTracer

Flow-based Diagnostics for Embedding Systems & Compression

Installation • Core Focus • AI Agent Integration

---

GradTracer is a specialized diagnostic library designed for Embedding-heavy Recommendation Systems (RecSys) and Model Compression.

Rather than competing with general-purpose loggers like TensorBoard or Weights & Biases, GradTracer tracks step-by-step training dynamics (dG/dt)—such as embedding drift, gradient oscillation, and exposure frequency—to diagnose silent failures (e.g., representation collapse) that traditional scalar metrics miss.

Crucially, every GradTracer module exports its findings as standardized causal JSON, allowing AI coding assistants (Cursor, Copilot, Antigravity) and custom IDE Extensions to automatically parse the diagnosis and apply statistically backed prescriptions.

🎯 Core Focus: RecSys & Compression

1. Embedding Dynamics Tracker & Bayesian Auto-Fix (v0.6+)

General DL diagnostics often fail for RecSys because embedding tables are highly sparse and suffer from popularity bias. EmbeddingTracker identifies:

• Zombie Embeddings: Items with high update velocity but oscillating gradient directions (failing to generalize).
• Dead Embeddings: Items suffering from severe cold-start or broken negative sampling.
• Popularity Bias: Exposure distribution skew (Gini/Entropy) that hurts long-tail coverage.
• Frequency-Aware Saliency: Normalizes update velocity by exposure frequency to identify truly important embeddings for pruning.
• ⚡ [NEW] Bayesian Auto-Fix: Pass auto_fix=True to let GradTracer actively intercept and dynamically scale gradients for Zombie embeddings during the backward pass based on SNR and global loss posteriors.
• 🌍 [NEW] DDP Support: Automatically handles torch.distributed.all_reduce to aggregate embedding stats across multiple GPUs.
• 🛡️ [NEW] White-Glass Audit: Complete transparency. The AutoFixAuditLogger saves every causal intervention locally to .gradtracer/audit.jsonl to ensure perfect reproducibility.

2. Production Hardening & Ecosystem (v0.7)

• 🏎️ Zero-Overhead Tracking: Lazy evaluation with track_interval and strictly GPU-bound tensor operations guarantee < 5% performance overhead even on massive 1M+ parameter embedding tables (Verified in examples/benchmark.py).
• 🌉 Weights & Biases Bridge: Use log_to_wandb(tracker) to seamlessly stream GradTracer's diagnostics (zombie ratios, health scores, Popularity Gini) directly into your existing W&B dashboards.
• 🤖 Universal JSON Export: Replaced legacy XML with standard causal JSON output for flawless integration with modern LLM coding agents.
• 📈 Human-Readable Visualizations: Beautiful Seaborn/Matplotlib 3-panel distributions to intuitively understand popular bias, dead neurons, and oscillating gradients at a glance.

3. Auto-Compression Suite & IDE Extension (v0.7)

Instead of blindly pruning based on weight magnitude, GradTracer uses training dynamics to guide compression:

• [NEW] 🥇 Auto-Compression Recipe (RecipeGenerator): The Holy Grail of compression. Analyzes dynamic health and SNR to automatically output an optimal mixed-precision (FP16/INT8/INT4) and joint structural pruning recipe (e.g., "INT4 + 80% Prune").
• [NEW] 💻 VS Code Extension: Integrates directly into your IDE. Hover over Python layer definitions to see real-time GradTracer diagnostic popups and 1-click apply compression recipes.

4. Deep Tree Dynamics (v0.6)

• Node-Level GBDT Tracking (TreeDynamicsTracker): Unlike basic feature importance, GradTracer unpacks the raw tree structure to track Leaf Velocity (Variance) and Feature Split Concentration. Evaluates mathematically if trees are stagnating or exploding.
• [NEW] Broad Support: Now fully supports XGBoost, LightGBM, and CatBoost.

5. Complex Architectures (v0.6)

• FlowManager: Centralized multi-tracker hub designed for Two-Tower, GNN, or Sequential architectures to calculate cross-layer correlations (e.g., User Tower vs. Item Tower).

🤖 AI Agent JSON Export

GradTracer serves as a "Decision Layer" for AI coding assistants. By calling AgentExporter.export_embedding(), models receive exact logic and prescriptions.

from gradtracer.agent import AgentExporter
from gradtracer.analyzers.embedding import EmbeddingTracker

tracker = EmbeddingTracker(model.item_emb, auto_fix=True, track_interval=100)
# ... training loop (tracker.step()) ...

print(AgentExporter.export_embedding(tracker))

{
  "gradtracer_embedding_report": {
    "layer": "item_emb",
    "embedding_stats": {
      "num_embeddings": 1000000,
      "active_coverage_pct": 20.4,
      "popularity_gini": 0.85
    },
    "diagnostics": [
      {
        "type": "POPULARITY_BIAS",
        "premise": "Exposure distribution is highly skewed (Gini coefficient: 0.85 > 0.8).",
        "implies": "Model is predominantly optimizing for top-popular items. Long-tail embeddings will suffer from inadequate learning.",
        "action": "APPLY_LOG_Q_CORRECTION_OR_INVERSE_FREQUENCY_SAMPLING",
        "expected_effect": "Debias the softmax logits and improve long-tail recommendation coverage.",
        "confidence": 0.95
      }
    ]
  }
}

📊 Mathematical & Statistical Validation

GradTracer's recommendations are backed by formal statistical tests. As demonstrated in our Validation Notebooks, our Auto-Fix logic and embedding prescriptions yield Statistically Significant Improvements mathematically guaranteed via:

• NDCG@10 & Hit Rate@10 Paired t-tests demonstrating statistically rigorous ranking improvements when Auto-Fix intercepts oscillatory parameters in MovieLens-100K MF baselines.
• Cosine Similarity Tracking guaranteeing true oscillatory embeddings rather than in-sample noise.

🚀 Quick Start

pip install gradtracer

🧩 Experimental Modules

While optimized for RecSys and Compression, GradTracer still includes its original Python dynamics modules:

• FlowTracker: General PyTorch training stability (SNR, Stagnation).
• FeatureAnalyzer: VIF-filtered interaction suggestions.

License

MIT