cf-basic-io

Cogniflow I/O StepPackage providing data source and sink steps.

Steps

Dev/demo steps (cfbd:InlineSource, cfbd:OpcuaPhSource) now live in cf-basic-dev. Data hive parquet persistence now lives in cf-basic-sinks as cfsink:DataHiveParquetSinkStep.

Pipeline backend operations

The package also registers pipeline backend operations:

Installation

pip install cf-basic-io

With optional dependencies:

pip install cf-basic-io[duckdb]   # DuckDB sink support
pip install cf-basic-io[all]      # All optional deps (currently duckdb)

Virtual OPC UA pH server

The demo OPC UA server lives in the separate cf-opcua-server package.

pip install cf-opcua-server
cf-opcua-server start --interval 0.5
# or:
python -m cf_opcua_server.cli start --interval 0.5
# server runs at opc.tcp://127.0.0.1:4840/VirtualPhServer with pH/temperature nodes under ns=2

cfbd:OpcuaPhSource (from cf-basic-dev) is a pure reader and does not start servers; run the demo server externally.

C++ Extensions

Native plugin surfaces for high-performance pipeline operations that stay owned by cf-basic-io:

• _duckdb_backend_ops - DuckDB checkpoint/resume (requires DuckDB C++ library)

Build with C++ extensions:

CF_BASIC_IO_ENABLE_DUCKDB_CPP=1 pip install .

cfio:OpcuaReaderStep no longer links an OPC UA SDK inside cf-basic-io. The step is implemented by the cf_basic_io plugin and consumes the injected owner surface exported by cf-opcua-server.

Usage

from cf_basic_io.steps import handle_sink

# Write to DuckDB
handle_sink({"value": 42}, "duckdb:///output.duckdb?table=results")

StepPackage Integration

This package registers with Cogniflow via the cogniflow.steps entry point and ships steps.nq as its sole manifest. The cf:StepPackage node declares cf:packageId cf.basic.io.

Publishing

cf_basic_io is published with the dedicated Windows workflow:

• Workflow: .github/workflows/cf_basic_io_windows_publish.yml
• Package directory: sandcastle/cf_basic_steps/cf_basic_io
• PyPI tag: cf-basic-io-v<version>
• TestPyPI tag: cf-basic-io-v<version>-test

The publish path validates the default package build. OPC UA execution now lives behind the runtime-owned cf-opcua-server boundary, so no open62541 toggle is required for cf-basic-io publishing.

Local preflight:

powershell -ExecutionPolicy Bypass -File scripts/mimic_windows_python_publish_workflow.ps1 `
  -WorkflowFile .github/workflows/cf_basic_io_windows_publish.yml `
  -PackageDir sandcastle/cf_basic_steps/cf_basic_io `
  -PythonExe py `
  -PythonVersion 3.14

Queue a dry-run dispatch:

powershell -ExecutionPolicy Bypass -File scripts/queue_windows_python_publish_workflow.ps1 `
  -WorkflowFile .github/workflows/cf_basic_io_windows_publish.yml `
  -PackageDir sandcastle/cf_basic_steps/cf_basic_io `
  -PublishTarget testpypi `
  -Ref main `
  -RequireLocalPass `
  -DryRun

Watch queued runs:

gh run list --workflow cf_basic_io_windows_publish.yml --limit 10
gh run watch --exit-status