cf-basic-io 0.1.7

Cogniflow basic I/O StepPackage with optional C++ acceleration.

cf-basic-io

Cogniflow I/O StepPackage providing data source and sink steps.

Steps

Step	Category	Description
cfio:HandleSinkStep	sink	Persist data to DuckDB, CSV, or other handles
cfio:OpcuaReaderStep	source	Read configured values from an OPC-UA server
cfio:JsonFieldSelectStep	structural	Extract one field from a JSON payload
cfio:TimestampToEpochStep	structural	Convert ISO-8601 timestamp to epoch seconds
cfio:EpochToTimestampStep	structural	Convert epoch seconds to ISO-8601 timestamp
cfio:ReportChartStep	sink	Render chart output for reporting
cfio:ReportTableStep	sink	Render table output for reporting
cfio:StatsRowStep	structural	Assemble structured row outputs

Dev/demo steps (cfbd:InlineSource, cfbd:OpcuaPhSource) now live in cf-basic-dev. Data hive parquet persistence now lives in cf-basic-sinks as cfsink:DataHiveParquetSinkStep.

Pipeline backend operations

The package also registers pipeline backend operations:

Operation	Type	Description
handle_sink	python	Persist via duckdb:// or csv:// handles
duckdb_checkpoint	cpp	Checkpoint pipeline state to DuckDB
duckdb_resume_from_checkpoint	cpp	Resume from checkpoint
duckdb_sql	cpp	Execute SQL on checkpoint data

Installation

pip install cf-basic-io

With optional dependencies:

pip install cf-basic-io[duckdb]   # DuckDB sink support
pip install cf-basic-io[all]      # All optional deps (currently duckdb)

Virtual OPC UA pH server

The demo OPC UA server lives in the separate cf-opcua-server package.

pip install cf-opcua-server
cf-opcua-server start --interval 0.5
# or:
python -m cf_opcua_server.cli start --interval 0.5
# server runs at opc.tcp://127.0.0.1:4840/VirtualPhServer with pH/temperature nodes under ns=2

cfbd:OpcuaPhSource (from cf-basic-dev) is a pure reader and does not start servers; run the demo server externally.

C++ Extensions

Optional native extensions for high-performance pipeline operations:

• _duckdb_backend_ops - DuckDB checkpoint/resume (requires DuckDB C++ library)
• _opcua_backend_ops - OPC-UA read via open62541

Build with C++ extensions:

CF_BASIC_IO_ENABLE_DUCKDB_CPP=1 pip install .
CF_BASIC_IO_ENABLE_OPCUA_CPP=1 pip install .

If you run the repo's factory reset script, native deps are staged in .native_deps/duckdb / .native_deps/open62541; the build now auto-detects those paths so a fresh install will attempt to compile the C++ ops without extra env vars.

Usage

from cf_basic_io.steps import handle_sink

# Write to DuckDB
handle_sink({"value": 42}, "duckdb:///output.duckdb?table=results")

StepPackage Integration

This package registers with Cogniflow via the cogniflow.steps entry point and ships steps.jsonld as its sole manifest. The cf:StepPackage node declares cf:packageId cf.basic.io.

Publishing

cf_basic_io is published with the dedicated Windows workflow:

• Workflow: .github/workflows/cf_basic_io_windows_publish.yml
• Package directory: sandcastle/cf_basic_steps/cf_basic_io
• PyPI tag: cf-basic-io-v<version>
• TestPyPI tag: cf-basic-io-v<version>-test

The initial publish path validates the default package build. Optional native extras such as OPC-UA support remain opt-in and are not required for the first publish workflow.

Local preflight:

powershell -ExecutionPolicy Bypass -File scripts/mimic_windows_python_publish_workflow.ps1 `
  -WorkflowFile .github/workflows/cf_basic_io_windows_publish.yml `
  -PackageDir sandcastle/cf_basic_steps/cf_basic_io `
  -PythonExe py `
  -PythonVersion 3.13

Queue a dry-run dispatch:

powershell -ExecutionPolicy Bypass -File scripts/queue_windows_python_publish_workflow.ps1 `
  -WorkflowFile .github/workflows/cf_basic_io_windows_publish.yml `
  -PackageDir sandcastle/cf_basic_steps/cf_basic_io `
  -PublishTarget testpypi `
  -Ref main `
  -RequireLocalPass `
  -DryRun

Watch queued runs:

gh run list --workflow cf_basic_io_windows_publish.yml --limit 10
gh run watch --exit-status