kailash 0.11.0

Python SDK for the Kailash container-node architecture

Kailash Python SDK

Enterprise-Grade Workflow Orchestration Platform

Build production-ready applications with zero-config database operations, multi-channel platforms, and comprehensive AI integration. From rapid prototyping to enterprise deployment.

---

🔥 Latest Release: v0.10.0 (October 26, 2024)

⚠️ BREAKING CHANGES - Runtime Parity & Parameter Scoping

🚨 Breaking Changes

1. AsyncLocalRuntime Return Structure

# Before (v0.9.x):
results = await runtime.execute_workflow_async(workflow, {})
user = results["create_user"]["id"]  # Nested dict

# After (v0.10.0):
results, run_id = await runtime.execute_workflow_async(workflow, {})
user = results["create_user"]["id"]  # Tuple unpacking required

2. Validation Exception Types

# Before: RuntimeExecutionError
# After: ValueError (for connection_validation mode errors)

3. Parameter Scoping (Automatic Unwrapping)

# Before: Manual parameter management required
parameters = {"node1": {"value": 10}}

# After: Automatic unwrapping and isolation
parameters = {
    "node1": {"value": 10},  # Auto-unwrapped for node1
    "node2": {"value": 20},  # Isolated to node2 only
    "global_key": "value"    # Shared across all nodes
}

✨ What's New

• Runtime Parity (100%): LocalRuntime and AsyncLocalRuntime now behave identically
• Parameter Scoping: Automatic unwrapping of node-specific parameters with cross-node isolation
• Smart Parameter Filtering: Global parameters shared, node-specific parameters isolated
• CI Performance: 10x faster test execution (removed coverage overhead)

📈 Impact

• API Consistency: Both runtimes return (results, run_id) tuple
• Parameter Safety: No more cross-node parameter leakage
• Test Coverage: 928/928 tests passing (100%)
• CI Speed: Shared test suite - 2.53s (was 20-30 min)

📦 Package Updates

• kailash: v0.10.0 - Breaking changes for runtime parity
• kailash-dataflow: Updated dependencies to kailash>=0.10.0

🔄 Migration Guide

See CHANGELOG.md for detailed migration instructions.

Full Changelog | Core SDK 0.10.0

🎯 What Makes Kailash Different

🏗️ Complete Application Framework

Not just a toolkit - complete production-ready applications built on enterprise-grade infrastructure:

• DataFlow: Zero-config database operations with MongoDB-style queries
• Nexus: Multi-channel platform (API + CLI + MCP) from single codebase
• AI Registry: Advanced RAG with 47+ specialized nodes
• User Management: Enterprise RBAC system with comprehensive security

🚀 Performance & Scale

• 11x faster test execution (117s → 10.75s) with smart isolation
• 31.8M operations/second query performance baseline
• 30,000+ iterations/second cyclic workflow execution
• 100% test pass rate across 928 tests

🤖 AI-First Architecture

• A2A Google Protocol for enterprise multi-agent coordination
• Real MCP execution by default for all AI agents
• 47+ specialized RAG nodes for document processing
• Semantic memory systems with context-aware retrieval
• Hybrid search algorithms for intelligent agent discovery
• Self-organizing agent pools with advanced coordination patterns

🏗️ Project Architecture

Three-Layer Architecture

┌─────────────────────────────────────────────────────────────────┐
│                     🎨 Studio UI Layer                         │
│              Visual workflow builder (coming soon)              │
└─────────────────────────────────────────────────────────────────┘
                                  │
┌─────────────────────────────────────────────────────────────────┐
│                  🏢 Application Framework                       │
│  DataFlow  │  Nexus  │  AI Registry  │  User Management  │...  │
└─────────────────────────────────────────────────────────────────┘
                                  │
┌─────────────────────────────────────────────────────────────────┐
│                     🎯 Core SDK Foundation                      │
│  115+ Nodes  │  Workflows  │  Runtime  │  Security  │  Testing │
└─────────────────────────────────────────────────────────────────┘

Current Project Structure

kailash_python_sdk/
├── src/kailash/          # Core SDK - 115+ nodes, workflows, runtime
├── apps/                 # Complete Applications
│   ├── kailash-dataflow/ # Zero-config database operations
│   ├── kailash-nexus/    # Multi-channel platform
│   ├── kailash-mcp/      # Enterprise MCP platform
│   ├── ai_registry/      # Advanced RAG capabilities
│   └── user_management/  # Enterprise RBAC system
├── tests/               # 928 tests (100% pass rate)
├── docs/                # Comprehensive documentation
└── examples/            # Feature validation examples

🚀 Quick Start

Installation Options

# Core SDK only
pip install kailash

# With complete app frameworks
pip install kailash[dataflow,nexus]  # Database + multi-channel
pip install kailash[all]             # Everything

# Or install apps directly
pip install kailash-dataflow  # Zero-config database framework
pip install kailash-nexus     # Multi-channel platform

DataFlow: Zero-Config Database Operations

from dataflow import DataFlow

# Zero-configuration database operations
app = DataFlow()

# MongoDB-style queries across any database
users = app.query("users").where({"age": {"$gt": 18}}).limit(10)

# Redis-powered caching with smart invalidation
cached_result = app.cache().get("user_stats",
    lambda: app.query("users").aggregate([
        {"$group": {"_id": "$department", "count": {"$sum": 1}}}
    ])
)

# Start enterprise API server
app.start()  # Automatic API generation, monitoring, health checks

Nexus: Multi-Channel Platform

from nexus import Nexus

# Single codebase → API + CLI + MCP
app = Nexus()

# Register workflow once, available on all channels
@app.workflow
def process_data(input_data):
    return {"processed": len(input_data)}

# Zero-config startup
app.start()

# Now available as:
# - REST API: POST /workflows/process_data
# - CLI: nexus run process_data
# - MCP: AI agents can call process_data tool

A2A Multi-Agent Coordination

from kailash.workflow.builder import WorkflowBuilder
from kailash.runtime.local import LocalRuntime

# Build A2A coordination workflow
workflow = WorkflowBuilder()
workflow.add_node("A2ACoordinatorNode", "coordinator", {
    "use_google_protocol": True,
    "enable_semantic_memory": True,
    "delegation_strategy": "skill_based"
})
workflow.add_node("HybridSearchNode", "discovery", {
    "strategies": ["semantic", "keyword", "skill_based"],
    "adaptive_optimization": True
})
workflow.add_node("SemanticMemoryNode", "memory", {
    "embedding_provider": "openai",
    "memory_type": "long_term",
    "context_window": 8192
})

# Connect for intelligent agent coordination
workflow.add_connection("coordinator", "discovery", "agent_request", "search_query")
workflow.add_connection("discovery", "memory", "agent_matches", "context")

# Execute with enterprise monitoring
runtime = LocalRuntime()
results, run_id = runtime.execute(workflow.build())

🎯 Key Features

🧪 Testing Excellence

• 928 tests with 100% pass rate
• 11x performance improvement (117s → 10.75s execution)
• Docker integration for real PostgreSQL, Redis, MongoDB
• Smart isolation without process forking overhead

🏢 Enterprise Ready

• Multi-tenant architecture with complete isolation
• RBAC/ABAC security with fine-grained permissions
• Audit logging with compliance frameworks (GDPR, CCPA)
• Distributed transactions with Saga/2PC patterns
• Circuit breaker and resilience patterns

🤖 AI Integration

• 115+ production nodes including 47+ specialized RAG nodes
• Real MCP execution by default for all AI agents
• Self-organizing agent pools with intelligent coordination
• Complete LLM integration with embeddings and vector search

⚡ Performance & Scale

• 31.8M operations/second query performance baseline
• Connection pooling with automatic resource management
• Redis caching with intelligent invalidation patterns
• Async/await throughout with proper separation

🏗️ Node Ecosystem (115+ Nodes)

Core Categories

• Data Nodes: CSVReaderNode, AsyncSQLDatabaseNode, QueryBuilderNode, QueryCacheNode
• AI Nodes: LLMAgentNode, IterativeLLMAgentNode, EmbeddingGeneratorNode, SelfOrganizingAgentNode
• A2A Nodes: A2ACoordinatorNode, HybridSearchNode, AdaptiveSearchNode, SemanticMemoryNode, StreamingAnalyticsNode
• RAG Nodes: 47+ specialized nodes for document processing and retrieval
• Security Nodes: ThreatDetectionNode, AuditLogNode, AccessControlManager
• Monitoring Nodes: TransactionMetricsNode, DeadlockDetectorNode, PerformanceAnomalyNode
• Transaction Nodes: DistributedTransactionManagerNode, SagaCoordinatorNode

Advanced Features

• A2A Communication: Google Protocol-based multi-agent coordination
• Semantic Memory: Long-term memory management for agent interactions
• Hybrid Search: Multi-strategy agent discovery and matching
• Cyclic Workflows: CycleBuilder API with convergence detection
• Distributed Transactions: Automatic Saga/2PC pattern selection
• Real-time Monitoring: WebSocket streaming with performance metrics
• Enterprise Security: Multi-factor auth, threat detection, compliance

📊 Performance Metrics

Recent Achievements

• 11x faster test execution: 117s → 10.75s with smart isolation
• 100% test pass rate: 928 tests across all categories
• 31.8M operations/second: Query performance baseline
• 30,000+ iterations/second: Cyclic workflow execution

Enterprise Benchmarks

• Query Cache: 99.9% hit rate with intelligent invalidation
• Connection Pooling: 10,000+ concurrent connections
• MCP Integration: 407 tests with 100% pass rate
• Security: Zero vulnerabilities in production deployment

🚀 Applications Built with Kailash

1. DataFlow - Zero-Config Database Platform (v0.3.3)

pip install kailash-dataflow

• MongoDB-style queries across PostgreSQL, MySQL, SQLite
• Redis caching with enterprise-grade invalidation
• Automatic API generation with OpenAPI documentation
• 4 production examples with complete deployment guides
• Latest: v0.3.3 - Critical connection parsing fix for special characters in passwords

2. Nexus - Multi-Channel Platform (v1.1.0)

pip install kailash-nexus

• Unified API, CLI, and MCP from single codebase
• Enterprise orchestration with multi-tenancy
• Session management with cross-channel synchronization
• 248 tests passing with comprehensive validation
• Latest: v1.1.0 - Production-ready with stub implementation fixes
  • Fixed 10 stub implementations (3 CRITICAL, 4 HIGH, 3 MEDIUM)
  • Event logging implemented (v1.0), real-time broadcasting planned (v1.1)
  • Metadata-based workflow schema extraction
  • Clear v1.0 vs v1.1 feature roadmap

3. AI Registry - Advanced RAG Platform

pip install kailash-ai-registry

• 47+ specialized RAG nodes for document processing
• Advanced retrieval with semantic search and re-ranking
• Multi-modal support with image and text processing
• Enterprise deployment with scalable architecture

4. User Management - Enterprise RBAC

pip install kailash-user-management

• Complete RBAC system with role hierarchy
• Multi-factor authentication with enterprise integration
• Audit logging with compliance frameworks
• Django-style capabilities built on SDK architecture

🧪 Testing & Quality

Comprehensive Test Suite

# All tests (928 tests - 100% passing)
pytest

# Fast unit tests (Tier 1)
pytest tests/unit/ -m 'not (slow or integration or e2e)' --timeout=1

# Parity tests (LocalRuntime vs AsyncLocalRuntime)
pytest tests/parity/ -v

# Shared runtime tests (both sync and async)
pytest tests/shared/runtime/ -v

# Integration tests with Docker
pytest tests/integration/ --timeout=5

# End-to-end scenarios
pytest tests/e2e/ --timeout=10

Test Infrastructure

• Docker Integration: Real PostgreSQL, Redis, MongoDB for testing
• Smart Isolation: Fixture-based isolation without process forking
• Performance Monitoring: Automated benchmarks and regression detection
• 100% Pass Rate: Comprehensive fixes across all test categories

🛡️ Security & Compliance

Enterprise Security

• Multi-factor Authentication: TOTP, WebAuthn, SMS integration
• Threat Detection: Real-time analysis with behavior monitoring
• Access Control: Fine-grained RBAC/ABAC with policy engines
• Audit Logging: Comprehensive trails with integrity verification

Compliance Frameworks

• GDPR/CCPA: Built-in data protection and privacy controls
• SOX: Financial reporting controls and audit trails
• HIPAA: Healthcare data protection patterns
• Multi-tenant Isolation: Complete tenant-aware operations

📚 Documentation & Resources

For Users

• SDK Users Guide: Complete workflow development guide
• Node Selection Guide: Smart node selection with decision trees
• Enterprise Patterns: Production deployment patterns
• API Documentation: Complete API reference

For Contributors

• SDK Contributors Guide: Internal SDK development
• Architecture Decisions: ADRs and design decisions
• Testing Guide: 3-tier testing strategy

Quick References

• Cheatsheet: 53 copy-paste patterns
• Common Mistakes: Error patterns and solutions
• Performance Guide: Optimization patterns

🚢 Production Deployment

Container Deployment

# Export workflow to container format
python -c "
from kailash.utils.export import export_workflow
export_workflow(workflow, 'production.yaml', format='kailash')
"

# Deploy to Kailash platform
kailash deploy production.yaml --environment prod

Monitoring & Observability

from kailash.visualization import DashboardServer

# Real-time monitoring dashboard
server = DashboardServer(port=8080)
server.start()
# Open http://localhost:8080 for live metrics

Enterprise Features

• Multi-tenant deployment with complete isolation
• Distributed transactions with automatic recovery
• Circuit breaker patterns for resilience
• Health monitoring with automated alerting

🤝 Contributing

We use a Claude Code-driven workflow for all development:

New Team Member?

Start Here → NEW_TEAM_MEMBER.md

Development Workflow

# Clone and setup
git clone https://github.com/integrum/kailash-python-sdk.git
cd kailash-python-sdk
uv sync

# Run tests (928 tests - 100% passing)
pytest tests/unit/ -m 'not (slow or integration or e2e)' --timeout=1  # Tier 1 unit tests
pytest tests/parity/ -v             # Runtime parity tests
pytest tests/shared/runtime/ -v     # Shared runtime tests
pytest tests/integration/ --timeout=5  # Integration tests
pytest tests/e2e/ --timeout=10      # End-to-end tests

# Code quality
black .
isort .
ruff check .

Claude Code Workflow

All project management through conversational AI:

• No manual TODO editing - Claude Code handles all updates
• No direct GitHub issues - Created through planning sessions
• All progress tracked - Through natural conversation

See Contributing Guide and sdk-contributors/CLAUDE.md.

📈 Project Status

✅ v0.8.4 - A2A Google Protocol Enhancement

• Advanced Agent Coordination: A2ACoordinatorNode with Google Protocol patterns
• Hybrid Search System: Multi-strategy agent discovery and matching
• Semantic Memory: Long-term memory management for agent interactions
• Real-time Analytics: Streaming performance monitoring for A2A workflows
• Backward Compatible: Seamless integration with existing implementations
• Production Ready: Enterprise-grade multi-agent coordination patterns

✅ v0.10.0 - Runtime Parity & Parameter Scoping (BREAKING CHANGES)

• Runtime Parity (100%): LocalRuntime and AsyncLocalRuntime behave identically
• Parameter Scoping: Automatic unwrapping and cross-node isolation
• API Consistency: Both runtimes return (results, run_id) tuple
• Testing Excellence: 928 tests with 100% pass rate
• CI Performance: 10x faster test execution (removed coverage overhead)

✅ v0.8.1 - Complete App Framework

• Complete Application Framework: DataFlow, Nexus, AI Registry, User Management
• PyPI Integration: All packages available with proper extras support
• Performance Breakthrough: 11x faster test execution
• Testing Excellence: 928 tests with 100% pass rate
• Enterprise Ready: Production deployment patterns

✅ v0.7.0 - Major Framework Release

• DataFlow Platform: Zero-config database operations
• Nexus Platform: Multi-channel orchestration
• AI Registry: Advanced RAG capabilities
• User Management: Enterprise RBAC system
• Testing Infrastructure: Docker integration, comprehensive validation

🚧 Roadmap

• Visual Workflow Builder: Studio UI for drag-and-drop workflow creation
• Advanced Analytics: ML-powered workflow optimization
• Cloud Integration: Native AWS/GCP/Azure deployment
• Mobile SDKs: iOS and Android workflow execution

📄 License

This project is licensed under the Apache License, Version 2.0. You may use, modify, distribute, and commercialize the software freely, subject to the terms of the license.

See the LICENSE file for the full license text.

The Kailash SDK is the subject of patent applications owned by Integrum Pte. Ltd. See the PATENTS file for details. Under Apache License 2.0, Section 3, each Contributor grants a patent license covering claims necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work, subject to the defensive termination clause in Section 3.

🙏 Acknowledgments

Built with ❤️ by the Integrum team for the Kailash ecosystem.

Special recognition for the 11x performance breakthrough and 100% test pass rate achieved through innovative engineering and comprehensive testing strategies.

---

Ready to build enterprise-grade applications?
Install from PyPI • Documentation • Examples • GitHub