grai-build 0.4.1

Declarative knowledge graph modeling tool inspired by dbt

grai.build

Schema-as-code for graph databases - Documentation like dbt, migrations for Neo4j

📘 What is grai.build?

grai.build brings dbt's documentation experience to graph databases - define your schema in YAML, generate beautiful docs, and manage migrations.

It manages your graph schema, not your data. You define entities and relations in YAML, and grai.build:

• ✅ Validates your schema for consistency
• ✅ Generates Cypher constraints and indexes
• ✅ Documents your graph structure automatically (like dbt docs)
• ✅ Tracks lineage with interactive visualizations
• ✅ Integrates with your CI/CD pipeline

What it's NOT:

• ❌ Not an ETL tool (use Airflow, Prefect, or dbt for data loading)
• ❌ Not a data transformation framework (dbt does this for SQL)
• ❌ Not a replacement for your existing data infrastructure

Think of it as:

• Like dbt: Declarative YAML definitions, beautiful documentation, lineage tracking
• Like Alembic/Flyway: Database migrations and schema management
• For graphs: Manages Neo4j schema while your pipelines handle data

🚀 Quick Start

Installation

pip install grai-build

Create Your First Project

# Initialize a new project
grai init my-graph-project
cd my-graph-project

# Validate and build
grai build

# Generate documentation (like dbt docs)
grai docs --serve

# Deploy schema to Neo4j
grai run --uri bolt://localhost:7687 --user neo4j --password secret

# Load sample data for local testing
grai run --load-csv --password secret

📂 Project Structure

my-graph-project/
├── grai.yml              # Project manifest
├── entities/
│   ├── customer.yml      # Entity definitions
│   └── product.yml
├── relations/
│   └── purchased.yml     # Relation definitions
└── target/               # Compiled output
    └── neo4j/
        └── compiled.cypher

📝 Example

Entity: entities/customer.yml

entity: customer
source: analytics.customers
keys: [customer_id]
properties:
  - name: customer_id
    type: string
  - name: name
    type: string
  - name: region
    type: string

Relation: relations/purchased.yml

relation: PURCHASED
from: customer
to: product
source: analytics.orders
mappings:
  from_key: customer_id
  to_key: product_id
properties:
  - name: order_id
    type: string
  - name: order_date
    type: datetime

Compile to Cypher

grai build

Output (target/neo4j/compiled.cypher):

// Create Customer nodes
MERGE (n:customer {customer_id: row.customer_id})
SET n.name = row.name,
    n.region = row.region;

// Create Product nodes
MERGE (n:product {product_id: row.product_id})
SET n.name = row.name;

// Create PURCHASED relations
MATCH (from:customer {customer_id: row.customer_id})
MATCH (to:product {product_id: row.product_id})
MERGE (from)-[r:PURCHASED]->(to)
SET r.order_id = row.order_id,
    r.order_date = row.order_date;

🎯 Features

• Declarative modeling - Define your graph schema in YAML (like dbt models)
• Schema validation - Catch errors before deployment
• Documentation generation - Beautiful HTML docs with grai docs (like dbt docs generate/serve)
• Lineage visualization - Interactive graph and Mermaid diagrams showing data flow
• Multi-backend support - Start with Neo4j, expand to Gremlin later
• CLI-first - Integrates into your CI/CD pipeline
• Type-safe - Built with Pydantic for robust validation
• Extensible - Easy to add custom backends and transformations

🏗️ Real-World Usage

Local Development

# 1. Define schema
vim entities/customer.yml

# 2. Validate
grai validate

# 3. Generate documentation
grai docs --serve  # Opens browser with interactive docs

# 4. Deploy schema
grai run --schema-only

# 5. Test with sample data
grai run --load-csv

Production Deployment

# .github/workflows/deploy-schema.yml
name: Deploy Graph Schema

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Validate Schema
        run: grai validate

      - name: Deploy to Production
        run: |
          grai run --schema-only \
            --uri ${{ secrets.NEO4J_URI }} \
            --user ${{ secrets.NEO4J_USER }} \
            --password ${{ secrets.NEO4J_PASSWORD }}

With Your ETL Pipeline

# Your Airflow DAG
from airflow import DAG
from airflow.operators.bash import BashOperator
from your_etl import load_customers_to_neo4j

dag = DAG('graph_pipeline')

# 1. grai.build ensures schema is up-to-date
deploy_schema = BashOperator(
    task_id='deploy_schema',
    bash_command='grai run --schema-only',
    dag=dag
)

# 2. Your ETL loads the actual data
load_data = PythonOperator(
    task_id='load_data',
    python_callable=load_customers_to_neo4j,
    dag=dag
)

deploy_schema >> load_data

📦 Architecture

grai/
├── cli/              # Typer-based CLI commands
├── core/
│   ├── models.py     # Pydantic models (Entity, Relation, Property)
│   ├── parser/       # YAML → Python models
│   ├── validator/    # Schema validation
│   ├── compiler/     # Generate Cypher/Gremlin
│   ├── loader/       # Execute against databases
│   └── utils/        # Shared utilities
└── templates/        # Project templates

🧪 Development

Setup

# Clone the repo
git clone https://github.com/asantora05/grai.build.git
cd grai.build

# Install with dev dependencies
pip install -e ".[dev]"

# Run tests
pytest

# Format code
black grai/
ruff check grai/

📖 Documentation

Generate beautiful, interactive documentation for your graph:

# Generate and serve documentation locally
grai docs --serve

# Generate to custom directory
grai docs --output ./my-docs

# Just generate (don't serve)
grai docs

The documentation includes:

• 📊 Project overview with stats
• 📦 Entity catalog with properties
• 🔗 Relation catalog with mappings
• 🕸️ Interactive graph visualization (D3.js)
• 🔄 Lineage diagrams (Mermaid.js)

For development guidance, check out the instructions.

🗺️ Roadmap

• Core Pydantic models
• YAML parser
• Schema validator
• Cypher compiler
• Neo4j loader
• CLI commands (init, build, validate, run, docs)
• Graph IR export (JSON)
• Documentation generation (dbt-style)
• Lineage visualization (Mermaid + D3.js)
• Schema versioning and migrations
• Graph visualization improvements
• Gremlin backend support
• Incremental sync

📊 Current Status

v0.3.2 - Feature-complete MVP with migrations

• ✅ Core Models - Pydantic models for Entity, Relation, Property
• ✅ YAML Parser - Parse and load entity/relation definitions
• ✅ Schema Validator - Validate references and mappings
• ✅ Cypher Compiler - Generate Neo4j constraints and indexes
• ✅ Neo4j Loader - Execute Cypher against Neo4j instances
• ✅ Documentation Generator - Interactive HTML docs (like dbt docs)
• ✅ Lineage Tracking - Visualize data flow and dependencies
• ✅ Graph Visualizer - D3.js and Cytoscape visualizations
• ✅ Build Cache - Incremental builds for faster iteration
• ✅ CLI Commands - Full command suite (init, build, validate, run, docs, etc.)

257 tests passing | High coverage across all modules

See it in action:

# Initialize example project
grai init my-project
cd my-project

# Generate and view documentation
grai docs --serve

🤝 Contributing

Contributions are welcome! This is an early-stage project, so there's plenty of room for improvement.

📄 License

MIT License - see LICENSE for details.

💡 Inspiration

This project is inspired by:

• dbt - Analytics engineering workflow
• SQLMesh - Data transformation framework
• Amundsen - Data discovery and metadata

---

Built with ❤️ for the graph database community