cf-pipeline-report

Minimal HTML report renderer for Cogniflow RDF pipeline manifests.

Published distribution name:

pip install cf-pipeline-report

Status: early scaffolding. The renderer outputs a static HTML shell that uses Observable Runtime (observablehq) to fetch a JSON payload and render a live pipeline view. The payload can be served by a small API so the report stays decoupled from the running pipeline.

Quick start (static JSON payload):

cf pipeline report-render sandcastle/cf_pipeline/cf_pipeline_engine/examples/opcua_fifo_avg_to_duckdb_parquet_triggered.nq
python -m http.server --directory sandcastle/cf_pipeline/cf_pipeline_engine/examples 8000

The report package accepts RDF manifests in N-Quads formats (.nq, .nquads).

Live API (DuckDB-backed):

cf pipeline report sandcastle/cf_pipeline/cf_pipeline_engine/examples/opcua_fifo_avg_to_duckdb_parquet_triggered.nq --duckdb path/to/pipeline.duckdb

Then open http://127.0.0.1:8765/ in the browser.

Multiple pipelines (directory of manifests):

cf pipeline report sandcastle/cf_pipeline/cf_pipeline_engine/examples --duckdb-dir path/to/duckdb

The server will expose one report per pipeline and look for .nq and .nquads manifests named <pipeline_id>.duckdb in the provided directory (or nested under <pipeline_id>/<pipeline_id>.duckdb).

Live tables/plots read from the DuckDB view pipeline_data created by the Parquet archive sink. Ensure the pipeline writes to DuckDB before opening the report, or the live sections will show empty data.

If no --duckdb/--duckdb-dir is provided, the server will try to resolve the DuckDB file from the manifest by reading pipelineId and baseDir parameters on the ParquetArchiveSinkStep. In this repo the recommended default is the contract-owned Data Hive output under the active workspace, not a consumer-owned filesystem convention. Treat the manifest and DataHive contracts as the source of truth for locating produced artifacts.

Publishing

cf_pipeline_report is published with the dedicated Windows workflow:

• Workflow: .github/workflows/cf_pipeline_report_windows_publish.yml
• Package directory: sandcastle/cf_pipeline/cf_pipeline_report
• PyPI tag: cf-pipeline-report-v<version>
• TestPyPI tag: cf-pipeline-report-v<version>-test

Local preflight:

powershell -ExecutionPolicy Bypass -File scripts/mimic_windows_python_publish_workflow.ps1 `
  -WorkflowFile .github/workflows/cf_pipeline_report_windows_publish.yml `
  -PackageDir sandcastle/cf_pipeline/cf_pipeline_report `
  -PythonExe py `
  -PythonVersion 3.14

Queue a dry-run dispatch:

powershell -ExecutionPolicy Bypass -File scripts/queue_windows_python_publish_workflow.ps1 `
  -WorkflowFile .github/workflows/cf_pipeline_report_windows_publish.yml `
  -PackageDir sandcastle/cf_pipeline/cf_pipeline_report `
  -PublishTarget testpypi `
  -Ref main `
  -RequireLocalPass `
  -DryRun