nim-audit 0.1.0

A professional tool for auditing NVIDIA NIM containers

nim-audit

Stop deploying NIM containers blind.

A professional CLI tool for auditing NVIDIA NIM containers before they hit production.

Installation • Quick Start • Commands • User Guide

---

Why nim-audit?

Ever upgraded a NIM container and had everything crash 10 minutes later?

nim-audit catches what humans miss:

• 🔄 Breaking changes between container versions
• 🎮 GPU compatibility issues before deployment
• ⚙️ Environment variable risks and impacts
• 📋 Policy violations that slip through review
• 🔍 Behavioral drift between model versions

One command before every upgrade. Never be surprised again.

---

Installation

pip install nim-audit

Requirements: Python 3.11+

For development:

git clone https://github.com/ashzak/nim-audit.git
cd nim-audit
pip install -e ".[dev]"

---

Quick Start

Compare versions before upgrading

nim-audit diff nvcr.io/nim/llama3:1.0.0 nvcr.io/nim/llama3:1.1.0

╭─────────────────────────── Diff Report ───────────────────────────╮
│ Source: nvcr.io/nim/llama3:1.0.0                                  │
│ Target: nvcr.io/nim/llama3:1.1.0                                  │
╰───────────────────────────────────────────────────────────────────╯

⚠️  Breaking Changes Detected
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
• API: /v1/completions response schema changed
• Config: NIM_MAX_BATCH_SIZE default: 4 → 8
• Requirement: Min GPU memory increased to 24GB

Check GPU compatibility

nim-audit compat --image nvcr.io/nim/llama3:1.1.0 --gpu A10

╭─────────────────────── Compatibility Report ────────────────────────╮
│ Image: nvcr.io/nim/llama3:1.1.0                                     │
│ Status: ✅ COMPATIBLE                                               │
╰─────────────────────────────────────────────────────────────────────╯

Lint your environment file

nim-audit env lint --env-file production.env

╭───────────────────── Environment Lint Report ───────────────────────╮
│ Status: ⚠️  WARN                                                    │
╰─────────────────────────────────────────────────────────────────────╯

┃ Severity ┃ Variable           ┃ Message                             ┃
┡━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ WARN     │ NIM_MAX_BATCH_SIZE │ Registry marks as increasing memory │

---

Commands

Command	Description
diff	Compare two NIM versions, detect breaking changes
config	Analyze configuration with impact assessment
compat	Check GPU and driver compatibility
lint	Validate against enterprise policies
fingerprint	Compare runtime behavioral signatures
cluster	Scan Kubernetes cluster compatibility
env	Environment variable analysis suite

---

diff

Compare two NIM container versions:

# Basic diff
nim-audit diff old:tag new:tag

# Only breaking changes
nim-audit diff old:tag new:tag --breaking-only

# JSON output for CI/CD
nim-audit diff old:tag new:tag --format json --output report.json

# Filter by category
nim-audit diff old:tag new:tag --category api

Categories: metadata, model, tokenizer, api, runtime, layer, config, environment

---

config

Analyze NIM configuration and environment variables:

# Analyze image configuration
nim-audit config --image nvcr.io/nim/llama3:1.1.0

# With your env file
nim-audit config --image nvcr.io/nim/llama3:1.1.0 --env-file prod.env

# Validate configuration
nim-audit config --image nvcr.io/nim/llama3:1.1.0 --env-file prod.env --validate

# Show all options including defaults
nim-audit config --image nvcr.io/nim/llama3:1.1.0 --all

---

compat

Check GPU and driver compatibility:

# Check specific GPU
nim-audit compat --image nvcr.io/nim/llama3:1.1.0 --gpu A100

# With driver version
nim-audit compat --image nvcr.io/nim/llama3:1.1.0 --gpu A10 --driver 535.104.05

# Auto-detect local GPU
nim-audit compat --image nvcr.io/nim/llama3:1.1.0 --detect

Supported GPUs: A10, A100, H100, L4, L40, L40S, T4, V100, A6000, RTX 4090, RTX 6000

---

lint

Validate containers against policies:

# Lint with built-in rules
nim-audit lint --image nvcr.io/nim/llama3:1.1.0

# Custom enterprise policy
nim-audit lint --image nvcr.io/nim/llama3:1.1.0 --policy enterprise.yaml

# Only show errors
nim-audit lint --image nvcr.io/nim/llama3:1.1.0 --severity error

Built-in rules:

• nim-001: Require version label
• nim-002: No root user
• nim-003: Require model name
• nim-004: Check exposed ports
• nim-005: No sensitive environment variables

Custom Policy Example

name: enterprise-policy
version: "1.0.0"

rules:
  - id: ent-001
    name: require-security-scan
    description: Image must have security scan label
    severity: error
    condition: labels['security.scan.status'] == 'passed'
    remediation: Run security scan and add label

  - id: ent-002
    name: max-batch-size
    description: Batch size must not exceed 64
    severity: warning
    condition: int(env.get('NIM_MAX_BATCH_SIZE', '1')) <= 64

---

fingerprint

Generate and compare behavioral signatures:

# Generate fingerprint from running container
nim-audit fingerprint --image nvcr.io/nim/llama3:1.0.0 \
                      --endpoint http://localhost:8000 \
                      --output v1.0.0.json

# Compare two fingerprints
nim-audit fingerprint compare v1.0.0.json v1.1.0.json --tolerance 0.05

---

cluster

Scan Kubernetes cluster for NIM compatibility:

# Scan cluster
nim-audit cluster --image nvcr.io/nim/llama3:1.1.0

# Specific context
nim-audit cluster --image nvcr.io/nim/llama3:1.1.0 --context production

---

env

Environment variable analysis tools:

# Lint environment file
nim-audit env lint --env-file prod.env

# Describe a variable
nim-audit env describe NIM_MAX_BATCH_SIZE

# Diff two env files
nim-audit env diff staging.env production.env

# List all known variables
nim-audit env registry-list

---

Library API

Use nim-audit programmatically:

from nim_audit import NIMImage, DiffEngine, ConfigAnalyzer, CompatChecker

# Load images
img1 = NIMImage.from_local("nvcr.io/nim/llama3:1.0.0")
img2 = NIMImage.from_local("nvcr.io/nim/llama3:1.1.0")

# Diff
engine = DiffEngine()
result = engine.diff(img1, img2)
for bc in result.report.breaking_changes:
    print(f"Breaking: {bc.title}")

# Config analysis
analyzer = ConfigAnalyzer()
result = analyzer.analyze(img2, env={"NIM_MAX_BATCH_SIZE": "64"})
for entry in result.report.entries:
    print(f"{entry.name}: {entry.impact.level}")

# Compatibility
checker = CompatChecker()
result = checker.check(img2, gpu="A100")
print(f"Compatible: {result.report.compatible}")

---

CI/CD Integration

Add nim-audit to your pipeline:

# .github/workflows/nim-validate.yml
name: NIM Validation

on:
  pull_request:
    paths:
      - 'k8s/nim/*.yaml'
      - '.env.nim'

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install nim-audit
        run: pip install nim-audit

      - name: Lint environment
        run: nim-audit env lint --env-file .env.nim

      - name: Check compatibility
        run: nim-audit compat --image ${{ vars.NIM_IMAGE }} --gpu A100

      - name: Policy check
        run: nim-audit lint --image ${{ vars.NIM_IMAGE }} --policy policies/enterprise.yaml

---

Configuration

Create ~/.nim-audit.yaml:

cache:
  enabled: true
  ttl: 3600

registry:
  ngc_api_key: ${NGC_API_KEY}

output:
  default_format: terminal
  color: true

lint:
  include_builtin: true
  fail_on_warning: false

# Shortcuts for common images
aliases:
  llama3: nvcr.io/nim/meta/llama3-8b-instruct:latest
  llama3-70b: nvcr.io/nim/meta/llama3-70b-instruct:latest

Environment Variables:

• NGC_API_KEY: NVIDIA NGC API key
• NIM_AUDIT_CONFIG: Config file override
• NIM_AUDIT_NO_COLOR: Disable colors

---

Documentation

• User Guide - Comprehensive usage documentation
• CLI Reference - All commands and options

---

Architecture

nim-audit/
├── src/nim_audit/
│   ├── cli/          # CLI commands (Typer)
│   ├── core/         # Core domain logic
│   │   └── env/      # Environment analysis
│   ├── extractors/   # Artifact extractors
│   ├── registry/     # Container registry clients
│   ├── renderers/    # Output formatters
│   ├── models/       # Pydantic data models
│   ├── knowledge/    # NIM knowledge base
│   └── utils/        # Utilities
└── tests/
    ├── unit/         # Unit tests
    └── integration/  # Integration tests

---

Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing)
3. Make your changes
4. Run tests (pytest)
5. Submit a pull request

---

License

MIT - Use it however you want.

---

Built to prevent production incidents.
_{Star ⭐ if this saved you from a 3am page.}