isA_Data

Cloud Data Analytics & Processing Center

🎯 Purpose

独立的数据分析云服务，处理数据处理、分析、可视化等任务。 从 isA_MCP/tools/services/data_analytics_service 迁移而来，提供 API 服务。

🏗️ Architecture

isA_Data/
├── app/
│   ├── main.py                    # FastAPI 主服务
│   ├── config.py                  # 配置管理
│   └── api/
│       ├── analytics.py           # 数据分析端点
│       ├── processing.py          # 数据处理端点
│       └── visualization.py       # 数据可视化端点
│
├── services/
│   ├── analytics/                 # [迁移自 isA_MCP]
│   │   ├── pandas_ops.py          # Pandas 操作
│   │   ├── statistical.py         # 统计分析
│   │   └── ml_analysis.py         # ML 分析
│   │
│   ├── processing/
│   │   ├── etl.py                 # ETL 处理
│   │   ├── cleaning.py            # 数据清洗
│   │   └── transformation.py      # 数据转换
│   │
│   └── storage/
│       ├── database.py            # 数据库操作
│       └── cache.py               # 缓存管理
│
├── core/
│   ├── logging.py
│   └── clients/
│       └── model_client.py        # ISA Model 客户端
│
├── deployment/
│   ├── docker/
│   │   └── Dockerfile
│   └── kubernetes/
│       └── deployment.yaml
│
├── tests/
│   └── test_analytics.py
│
├── requirements.txt
└── README.md

📡 API Endpoints

Data Analytics

POST /api/v1/data/analyze
POST /api/v1/data/statistics
POST /api/v1/data/ml/predict

Data Processing

POST /api/v1/data/clean
POST /api/v1/data/transform
POST /api/v1/data/aggregate

Data Visualization

POST /api/v1/data/visualize/chart
POST /api/v1/data/visualize/dashboard

📦 Install (PyPI)

isa-data is published to PyPI — install it as an SDK to consume the canonical data-product contracts and the lifted data-product DevEx engines:

pip install "isa-data>=1.0.1"

Use >=1.0.1: 1.0.0 eagerly imported intelligent_query_service → deltalake (not a dependency), so the lightweight dataphin engines failed to import; #457 made that import lazy.

Data-product DevEx kit (consumed by towers, e.g. sn_commercial_tower)

The SDK ships the tower-agnostic engines a tower delegates to (see docs/data-products/tower-onboarding.md and ADR-0006):

from isa_data.contracts.data_products import build_artifact_inventory, ArtifactSurfaceSpec
from isa_data.services.data_fabric_service.federation.dataphin.scaffolding import scaffold
from isa_data.services.data_fabric_service.federation.dataphin.handoff import generate_handoff
from isa_data.services.data_fabric_service.federation.dataphin.deploy import build_plan_from_contract
from isa_data.services.product_spec.deploy_contract import load_ct_contract

Privileged Data Product stages use the canonical secret-free authorization contract and group lifecycle client documented in docs/data-products/access-authorization.md:

from isa_data.contracts.data_products import compile_access_intent
from isa_data.services.data_product_authorization import DataProductAuthorizationClient

These import without the heavy data-infra stack (deltalake/duckdb) — that is pulled only if you use IntelligentQueryService.

🚀 Usage

As Standalone Service

# Start server (FastAPI app)
uvicorn isa_data.main:app --host 0.0.0.0 --port 8084

# Or with Docker
docker-compose up

Called from isA_MCP

# isA_MCP 中通过 API 调用
from tools.services.data_analytics_service import ISA_Data_Client

client = ISA_Data_Client(base_url="http://localhost:8001")
result = await client.analyze_data(
    data=df,
    analysis_type="descriptive"
)

💡 Benefits

1. 专注: 专门处理数据相关任务
2. 性能: 可以部署在大内存机器上处理大数据
3. 隔离: 数据处理故障不影响 MCP 主服务
4. 可复用: 多个服务都可以调用数据分析能力
5. 扩展: 未来可以添加更多数据源和分析能力

🔄 Migration Status

• Phase 1: 基础架构搭建
• Phase 2: 迁移 data_analytics 模块
• Phase 3: FastAPI 端点实现
• Phase 4: 数据库集成
• Phase 5: isA_MCP 集成测试
• Phase 6: 生产部署

🚢 Deploying

The isa-data image ships only through the isA target-native release control plane (/cicd): catalog entry isa-data in isA_Cloud/deployments/release/platform-services.yaml. On gcp a VERIFIED build can open and merge its GitOps pull request when migration requirements are satisfied. The controller computes migration-noop evidence only when the migration tree is unchanged; a changed tree requires the attended migration evidence. On the enterprise sn target promotion needs landed migration evidence.

# Name the blocking class (GITHUB, DEVICE, PAM, VAULT, ...) and who must act
python3 "$ISA_SKILLS_ROOT/skills/cicd/scripts/delivery-readiness.py" isa-data --target gcp
isa-cicd submit isa-data --target gcp --revision isA_Data=<40-char landed SHA>

Blocker guidance lives in isa_skills skills/cicd/references/blockers.md. The GitHub Actions workflows deploy.yml (build + kubectl set image) and docker.yml (GHCR push) are not the target-native release path. deployment/ contains local development files as well as older production/reference manifests; their direct-apply examples do not replace the target's catalog-owned GitOps promotion and deployed-version readback.

Publishing the isa-data service or Python SDK does not publish a customer's collection task. A consuming implementation owns its source definitions, account bindings, task/config revisions, worker image and schedules. For Commercial Tower, follow that repository's deployment/collectors/README.md: publish the immutable config package and target GitOps pin, release a signed worker image when code changes, then verify the deployed revision and a bounded RAW→STD→consumer run. Recurring collection additionally requires the governed scheduler; exporting a schedule or running np collect once does not install one. CDC uses its supported continuous runtime rather than the one-shot collector Job.

🔗 Related Projects

• isA_MCP: MCP 工具层（API 调用端）
• isA_OS: Web & OS 操作云服务
• isA_Model: 模型服务