ashmatics-aigov-tools

Version: 0.7.4 Last Updated: 2026-07-12 Package: ashmatics-aigov-tools (renamed from ashmatics-cai-framework in v0.7.1)

See CHANGELOG.md for full version history.

A Python package for managing, validating, and deploying the Ashmatics Clinical AI Governance Framework content.

Overview

This package provides tools for the complete lifecycle management of the Ashmatics CAI Governance Framework, including:

• Content Validation: Verify framework structure and integrity
• Azure Storage Management: Upload and version framework content
• Database Registration: Register content with PostgreSQL (Hasura) and MongoDB
• Hybrid Persistence: Implement three-layer architecture (Azure + Postgres + MongoDB)
• MongoDB Compilation: Convert Markdown content into structured JSON views for AI agents

Architecture - Hybrid Persistence Strategy

The package implements a three-layer hybrid persistence approach:

1. Azure Blob Storage (Source of Truth)

• Immutable Markdown source files with content-addressed IDs
• Versioned storage with v{version}/ prefixes
• Human-readable format for web presentation

2. PostgreSQL via Hasura (Relational Metadata)

• Base artifact metadata and version tracking
• File registry for search and organization
• Tables: framework_content_registry, framework_content_registry_files

3. MongoDB Atlas (forge_kb — Compiled Views + Vectors for AI Agents)

• Structured JSON views with sections, headers, placeholders — aigov_framework_content
• Policy bindings and traceability information
• Per-section embeddings for semantic search — aigov_framework_vectors with Atlas Vector Search index vs_aigov_framework_vectors (queried by the KB reader's $vectorSearch)

Installation

From Git Repository (Private)

# Using pip
pip install git+https://github.com/AshMatics/ashmatics-aigov-tools.git

# Using uv
uv add git+https://github.com/AshMatics/ashmatics-aigov-tools.git

# With development dependencies
pip install "ashmatics-aigov-tools[dev] @ git+https://github.com/AshMatics/ashmatics-aigov-tools.git"

From Local Development

# Clone the repository
git clone https://github.com/AshMatics/ashmatics-aigov-tools.git
cd ashmatics-aigov-tools

# Install in editable mode with dev dependencies
pip install -e ".[dev]"

Using uv (Recommended)

For local development, we recommend using uv for faster, more reliable dependency management:

# Install uv if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh

# Clone and set up the project
git clone https://github.com/AshMatics/ashmatics-aigov-tools.git
cd ashmatics-aigov-tools

# Sync dependencies (creates/updates .venv automatically)
uv sync

# Run commands using the project's isolated environment
uv run register-framework-content --version=0.7.0 --hybrid --dry-run

Important: When using uv, always prefix commands with uv run to ensure they execute in the project's virtual environment rather than your global Python installation.

Dependencies

This package depends on ashmatics-tools for shared utilities:

• Document parsing (Docling, LlamaParse)
• Chunking strategies
• Embedding generation
• Vector storage
• Base importers and processors

Usage

Command Line Interface

Register Existing Framework Content

# Register existing Azure content with PostgreSQL only
uv run register-framework-content --version=0.7.0

# Full hybrid registration (PostgreSQL + MongoDB)
uv run register-framework-content --version=0.7.0 --hybrid

# Dry run to see what would be registered
uv run register-framework-content --version=0.7.0 --hybrid --dry-run

# Validate artifact ID mapping consistency
uv run register-framework-content --version=0.7.0 --hybrid --validate-mapping

Note: If not using uv, you can run commands directly after installation, but uv run is recommended for local development to ensure proper environment isolation.

Generate Embeddings (cai-generate-embeddings)

Embeds the compiled framework views at one specific version into aigov_framework_vectors, one vector per section (text-embedding-3-small, 1536-dim), and ensures the Atlas Vector Search index vs_aigov_framework_vectors exists and is READY — the index the KB reader's semantic_search_framework queries. Every vector carries framework_version + embedding_model. Environment is the forge contract (forge.env: MONGO_URL, MONGO_DB, AZURE_OPENAI_*). --version is required — there is deliberately no default, because it selects which documents are embedded and a wrong value fails silently.

# Stage / prod: every markdown artifact (WP- included) at the served version
AZURE_OPENAI_DEPLOYMENT_NAME=text-embedding-3-small \
  uv run cai-generate-embeddings --db forge_kb --version 0.10.0 --format md

# Vectors-by-version + index status (read-only)
uv run cai-generate-embeddings --stats-only --db forge_kb

# Create / verify the Atlas index on existing vectors (no embedding)
uv run cai-generate-embeddings --index-only --db forge_kb

# Re-embed one slice: delete its old vectors, then embed it (one command)
uv run cai-generate-embeddings --db forge_kb --version 0.10.0 \
  --clear-prefix SOP-PV --include SOP-PV --format md

The script refuses to run if AZURE_OPENAI_DEPLOYMENT_NAME names a different model than text-embedding-3-small (two 1536-dim spaces pass every count check and return nonsense). In a full reseed this runs as the graph-seed char-embeddings step in ashmatics-knowledgebase-tools (ASHFORGE-347), which pins the deployment and the version.

Key Options:

Query the AI Governance Framework (RAG)

The cai-query tool provides RAG-powered question answering over the AI Governance Framework content. It retrieves over the KB MCP server (semantic_search_framework against aigov_framework_vectors) and generates answers using Azure OpenAI or Ollama.

# Interactive mode (recommended for exploration)
uv run cai-query

# Single query
uv run cai-query --query "What is performance validation?"

# Use Ollama instead of Azure OpenAI
uv run cai-query --ollama --model llama3.2

# Multi-query RAG for better retrieval coverage
uv run cai-query --multi-query --query "Explain risk management"

# Stream the response
uv run cai-query --stream --query "What are governance controls?"

# Retrieve more sources
uv run cai-query --top-k 10 --query "What is the PV process?"

Key Options:

Interactive Commands:

Environment Variables (forge.env):

Extract Framework Relationships (Graph)

The cai-extract-relationships tool extracts relationships between framework artifacts for graph database integration (ASHKBAPP-60).

# Extract relationships to JSON
uv run cai-extract-relationships --output relationships.json

# Preview without writing
uv run cai-extract-relationships --dry-run

Initialize ASHCAI Ontology

The cai-init-ontology tool initializes the ASHCAI (AshMatics Clinical AI Governance) ontology collections in MongoDB, loading policy domains, process domains, and relationships from the ashmatics-framework-registry.json file (the single source of truth).

# Initialize with auto-discovered registry file
uv run cai-init-ontology

# Initialize with explicit registry file path
uv run cai-init-ontology --registry-file /path/to/ashmatics-framework-registry.json

# Dry run to preview what would be created
uv run cai-init-ontology --dry-run

# Force re-initialization (updates existing data)
uv run cai-init-ontology --force

# Custom MongoDB connection
uv run cai-init-ontology --mongodb-uri "mongodb://localhost:27017" --database "ashmatics_dev"

Key Options:

Registry Auto-Discovery: The registry file is automatically discovered from:

• Sibling ashmatics-policy-process-builder repository
• ~/GitHub/AsherInformatics/ashmatics-policy-process-builder/
• /app/config/ (for Docker deployments)

What Gets Loaded:

• 12 Policy Domains: SAP, AGP, RMP, HOP, EFP, MMP, LTP, CMP, DGP, SEP, VAL, TPP
  • Includes: tier, category, ISO 42001 controls, exemplar controls, regulatory mappings
• 12 Process Domains: ORG, OVR, PV, SA, IT, RM, MON, HO, EF, TR, UG, SE
  • Includes: base practices, policy bindings, upstream/downstream dependencies
• 51 Relationships: Policy→Process "specifies" relationships from policyToProcess mappings

Collections Created:

• ashcai_policy_domains - Policy domain definitions (MMP-001, AGP-001, etc.)
• ashcai_process_domains - Process domain definitions (MON, RM, SA, etc.)
• ashcai_base_practices - Base practices (MON.BP01, etc.)
• ashcai_sop_templates - SOP templates (SOP-MON-01, etc.)
• ashcai_work_product_templates - Work product templates
• ashcai_exemplar_controls - Exemplar controls (EXC-A4-01, etc.)
• ashcai_regulatory_requirements - Regulatory requirements
• ashcai_regulatory_frameworks - Framework metadata
• ashcai_relationships - Entity relationships (policy→process, etc.)
• ashcai_regulatory_crosswalks - Regulatory crosswalk mappings

Sync Regulatory Crosswalks

The cai-sync-regulatory tool synchronizes regulatory crosswalk mappings from YAML source files (in ashmatics-policy-process-builder/wizard-content-schemas/) into the ASHCAI ontology collections.

# Sync all regulatory frameworks
uv run cai-sync-regulatory

# List available frameworks
uv run cai-sync-regulatory --list

# Sync specific frameworks only
uv run cai-sync-regulatory --frameworks NIST-AI-RMF JC-RUAIH

# Dry run to preview
uv run cai-sync-regulatory --dry-run

# Force full resync (delete existing crosswalks first)
uv run cai-sync-regulatory --force

# Custom YAML directory
uv run cai-sync-regulatory --yaml-dir /path/to/wizard-content-schemas

Key Options:

Supported Regulatory Frameworks:

Framework Explorer Pipeline

Added 2026-01-28 (ASHPSTUDIO-184)

The Framework Explorer Pipeline provides intelligent query processing for the ASHCAI AI Governance Framework. It orchestrates four stages to deliver accurate, well-cited responses:

Quick Start:

from ashmatics_aigov_tools.services import (
    FrameworkExplorerPipeline, KBMCPClient, MCPClientConfig, PipelineConfig
)

async with KBMCPClient(MCPClientConfig.from_env()) as client:
    pipeline = FrameworkExplorerPipeline(client, PipelineConfig())
    result = await pipeline.explore("What is Risk Management?")
    print(result.response)  # Synthesized answer
    print(result.citations)  # Supporting evidence

Supported Query Types:

• Domain explanations ("What is Risk Management?")
• Structural queries ("What SOPs implement MMP-001?")
• Regulatory crosswalks ("How does ASHCAI map to NIST AI RMF?")
• Gap analysis ("What controls are we missing for NIST?")
• Known item lookup ("Show me SOP-PV-01")
• Comparative analysis ("How is ASHCAI different from ISO 42001?")

Full documentation: See services/README.md for configuration, architecture details, and API reference.

AIGov Query Service SDK

Added 2026-01-25 (ASHPSTUDIO-184)

The services/ module provides a reusable SDK for querying the AI Governance Framework. This SDK is designed to be imported by applications (aigov-explorer, coreapp, etc.) without pulling in CLI dependencies.

Architecture

┌─────────────────────────────────────────────────────────────┐
│  Your Apps (aigov explorer, coreapp, etc.)                  │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐          │
│  │  FastAPI    │  │   Flask     │  │  Streamlit  │          │
│  │  routes     │  │   views     │  │   app       │          │
│  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘          │
└─────────┼────────────────┼────────────────┼─────────────────┘
          │                │                │
          ▼                ▼                ▼
┌─────────────────────────────────────────────────────────────┐
│  ashmatics_aigov_tools.services (SDK)                     │
│  ┌─────────────────────────────────────────────────────┐    │
│  │  AIGovQueryService                                  │    │
│  │  - query(question) -> QueryResult                   │    │
│  │  - search(question) -> RetrievalResult              │    │
│  │  - stream_query(question) -> AsyncIterator          │    │
│  │  - get_policy_hierarchy(policy_id) -> dict          │    │
│  └─────────────────────────────────────────────────────┘    │
│                         │                                   │
│                         ▼                                   │
│                  ┌──────────────┐                           │
│                  │  MCPBackend  │                           │
│                  │  (KB server) │                           │
│                  └──────────────┘                           │
└─────────────────────────────────────────────────────────────┘

Quick Start

from ashmatics_aigov_tools.services import AIGovQueryService, AIGovQueryConfig

# Create configuration (MCP mode - recommended for production)
config = AIGovQueryConfig(
    mcp_url="http://localhost:8088",  # or https://kb-api.ashmatics.com
    mcp_api_key=os.getenv("MCP_API_KEY"),
)

# Use the service
async with AIGovQueryService(config) as service:
    # Full RAG query (retrieval + LLM synthesis)
    result = await service.query("What SOPs for model monitoring?")
    print(result.answer)
    for source in result.sources:
        print(f"  - {source.document_id}: {source.score:.3f}")

    # Search only (no LLM synthesis)
    retrieval = await service.search("performance validation", top_k=5)
    for r in retrieval.results:
        print(f"{r.document_id}: {r.title}")

    # Streaming response (for real-time UIs)
    async for chunk in service.stream_query("Explain risk management"):
        print(chunk.text, end="", flush=True)

    # Graph traversals (MCP mode only)
    hierarchy = await service.get_policy_hierarchy("POL-MMP-001")
    controls = await service.find_control_implementations("EXC-A4-01")
    crosswalk = await service.get_regulatory_crosswalk("NIST-AI-RMF")

Configuration Options

Environment Variables

# MCP mode
MCP_URL=https://kb-api.ashmatics.com
MCP_API_KEY=coreapp_your_key_here

# Embedding generation (cai-generate-embeddings)
MONGO_URL=mongodb+srv://...
AZURE_OPENAI_API_KEY=...
AZURE_OPENAI_ENDPOINT=...

Backend Modes

MCP Backend (the only backend since 0.8.0)

• Uses KB MCP server for intelligent search routing
• Supports intent classification and suggested workflows
• Enables graph traversals (policy hierarchy, control implementations, regulatory crosswalks)
• Production-ready with proper auth

Future: SDK Extraction

Note: This SDK may be extracted to a separate ashmatics-kb-sdk package in the future to provide a lightweight client library without the framework management dependencies. This would allow apps to import just the query functionality without Azure blob, pymongo, and other admin tooling.

Programmatic Usage (Framework Management)

from ashmatics_aigov_tools.importers.framework_content import FrameworkContentImporter

# Initialize importer with MongoDB support
importer = FrameworkContentImporter(
    framework_repo_path="/path/to/framework",
    azure_connection_string="DefaultEndpointsProtocol=https;...",
    kb_table_name="framework_content_registry",
    container_name="framework-content",
    mongo_connection_string="mongodb+srv://user:pass@cluster/...",
    mongo_database="ashmatics_kb",
    mongo_collection="framework_compiled_views",
    graphql_endpoint="https://kb-graphql.ashmatics.com/v1/graphql",
    admin_secret="your-admin-secret"
)

# Hybrid registration (both Postgres and MongoDB)
success, results = importer.register_full_hybrid_content(
    force_refresh=False,
    verify_azure_content=True
)

# Postgres-only registration
success, results = importer.register_existing_content_with_kb(
    force_refresh=False,
    verify_azure_content=True
)

# MongoDB compilation only
success, results = importer.compile_and_store_mongo_views(
    force_refresh=False,
    verify_azure_content=True
)

Framework Content Structure

The framework includes:

• 12 Process Domains: ORG, OVR, PV, SA, IT, RM, MON, HO, EF, TR, UG, SE
• 12 Policy Domains: SAP, AGP, RMP, HOP, EFP, MMP, LTP, CMP, DGP, SEP, VAL, TPP
• ~50 Base Practices: Distributed across process domains
• 51 Policy→Process Relationships: Defining which policies govern which processes
• Content Categories:
  • domain_definitions - Process domain JSON definitions
  • policy_templates - Customizable policy JSON templates
  • process_documentation - Detailed process and procedure markdown files
  • master_registry - Framework registry and metadata files
  • validation_tools - Framework validation and tooling files
  • framework_documentation - General framework documentation

Artifact Slug Naming Convention

Added 2025-12-19

The framework uses human-friendly artifact_id slugs for API lookups instead of UUID-based identifiers. This enables clean API calls like:

GET /governance/POL-AGP-Overview?version=0.8.0
GET /governance/SOP-PV-01?version=0.8.0

Slug Prefixes by Artifact Type

Programmatic Usage

from ashmatics_aigov_tools import get_artifact_slug, validate_slug

# Generate slug from file path
slug = get_artifact_slug("policy-domain/AGP-Domain-Overview-Summary.md")
# Returns: "POL-AGP-Overview"

# Validate a slug
is_valid, error = validate_slug("SOP-PV-01")
# Returns: (True, "")

The slug mapping is defined in artifact_slug_mapping.py and is automatically used by the framework importer when registering content.

Components

Content Validator (processors/content_validator.py)

• Domain JSON file validation
• Policy logic consistency checks
• Process model documentation verification
• Master registry validation
• Version detection

Azure Uploader (processors/azure_uploader.py)

• Versioned blob uploads with structured paths
• Rich metadata tagging by content category
• File hash verification and deduplication
• Upload manifest generation
• ADLS Gen2 support

MongoDB Compiler (processors/mongo_compiler.py)

• Extracts sections, headers, and document structure
• Identifies placeholders/tokens for customization
• Parses policy bindings and process references
• Extracts traceability information and citations
• Generates schema-bound JSON for AI agents

Framework Content Importer (importers/framework_content.py)

Main orchestrator supporting:

1. Content validation
2. Azure blob upload with versioning
3. PostgreSQL registration (via Hasura GraphQL)
4. MongoDB compilation (compiled JSON views)
5. Version tracking across systems

Configuration

Environment Variables

Create a .env file with:

# Azure Storage
AZURE_STORAGE_CONNECTION_STRING="DefaultEndpointsProtocol=https;..."
AZURE_FRAMEWORK_CONTAINER="framework-content"

# PostgreSQL/Hasura
HASURA_GRAPHQL_ENDPOINT="https://kb-graphql.ashmatics.com/v1/graphql"
HASURA_ADMIN_SECRET="your-admin-secret"

# MongoDB Atlas (forge.env contract)
MONGO_URL="mongodb+srv://user:pass@forge-<env>-pl-0.../..."
MONGO_DB="forge_kb"
MONGO_COLLECTION="aigov_framework_content"

# Framework Source
FRAMEWORK_REPO_PATH="/path/to/framework/repo"

# Optional
FORCE_ADLS_GEN2="false"

Development

Running Tests

Using uv (recommended):

# Sync dev dependencies
uv sync --all-extras

# Run tests
uv run pytest

# Run with coverage
uv run pytest --cov=ashmatics_aigov_tools --cov-report=html

Or using pip:

# Install dev dependencies
pip install -e ".[dev]"

# Run tests
pytest

# Run with coverage
pytest --cov=ashmatics_aigov_tools --cov-report=html

Code Quality

# Using uv (recommended)
uv run ruff format src/
uv run ruff check src/
uv run mypy src/

# Or directly if tools are installed globally
ruff format src/
ruff check src/
mypy src/

Version Management

The framework uses semantic versioning with timestamp format: YYYY.MM.DD-HHMMSS

Example: 2025.11.20-153000

License

Copyright 2025 Asher Informatics PBC

Licensed under the Apache License, Version 2.0. See LICENSE file for details.

Related Projects

• ashmatics-tools - Shared utilities and primitives
• ashmatics-knowledgebase-tools - Knowledge base integration tools
• ashmatics-coreapp - Main application frontend

Changelog

See CHANGELOG.md for full version history.

Contributing

This is a private repository for Asher Informatics PBC. Contact the maintainers for access.

Support

For questions or issues, please contact:

• Email: john@asherinformatics.com
• GitHub Issues: https://github.com/AshMatics/ashmatics-aigov-tools/issues