cf-basic-sinks 0.1.3

Cogniflow basic sink StepPackage with C++ data hive parquet sink.

cf-basic-sinks

cf-basic-sinks provides native sink steps for pipeline outputs.

Steps

Step	Category	Description
cfsink:DataHiveParquetSinkStep	sink	Write canonical data hive parquet output through the cf_datahive_cpp gatekeeper

Published distribution name:

pip install cf-basic-sinks

Native build prerequisites

cf_basic_sinks is built with scikit-build-core and CMake and requires:

• CPython 3.13
• CMake on PATH
• a Windows C++ toolchain compatible with that CMake installation
• access to cf-pipeline-sdk from the package index
• access to cf-datahive from the package index for the owner-provided native cf_datahive_cpp source surface
• DuckDB native headers/import library/runtime (duckdb.dll) for shared-link builds

This package does not publish cf_datahive_cpp separately. The step package consumes the owner-provided native source surface exposed by the installed cf_datahive package. Native data hive write ownership remains under cf_datahive.

Shared-link mode can be selected with CF_DATAHIVE_CPP_DUCKDB_LINKAGE=shared. In that mode, DuckDB inputs are satisfied through either:

• the repo-local .native_deps/duckdb layout created by the setup scripts
• CF_DATAHIVE_CPP_DUCKDB_INCLUDE, CF_DATAHIVE_CPP_DUCKDB_LIB, and optionally CF_DATAHIVE_CPP_DUCKDB_DLL

Publishing

cf_basic_sinks is published with the dedicated Windows workflow:

• Workflow: .github/workflows/cf_basic_sinks_windows_publish.yml
• Package directory: sandcastle/cf_basic_steps/cf_basic_sinks
• PyPI tag: cf-basic-sinks-v<version>
• TestPyPI tag: cf-basic-sinks-v<version>-test

Workflow note:

• test/build jobs provision native deps via scripts/setup_native_deps_v2.ps1
• test/build jobs preinstall published cf-pipeline-sdk and cf-datahive into the job interpreter before invoking the shared publish helper, so the CMake Python subprocesses can discover the owner-provided SDK and data-hive surfaces
• test/build jobs export CF_DATAHIVE_CPP_DUCKDB_LINKAGE=shared and resolve DuckDB from the repo-local .native_deps/duckdb layout
• test/build jobs also export CF_DATAHIVE_CPP_DUCKDB_INCLUDE, CF_DATAHIVE_CPP_DUCKDB_LIB, and CF_DATAHIVE_CPP_DUCKDB_DLL

Local preflight:

$env:CF_DATAHIVE_CPP_DUCKDB_LINKAGE = "shared"
$env:CF_DATAHIVE_CPP_DUCKDB_INCLUDE = (Resolve-Path .native_deps/duckdb/include).Path
$env:CF_DATAHIVE_CPP_DUCKDB_LIB = (Resolve-Path .native_deps/duckdb/lib/duckdb.lib).Path
$env:CF_DATAHIVE_CPP_DUCKDB_DLL = (Resolve-Path .native_deps/duckdb/bin/duckdb.dll).Path
powershell -ExecutionPolicy Bypass -File scripts/mimic_windows_python_publish_workflow.ps1 `
  -WorkflowFile .github/workflows/cf_basic_sinks_windows_publish.yml `
  -PackageDir sandcastle/cf_basic_steps/cf_basic_sinks `
  -PythonExe py `
  -PythonVersion 3.13

Queue a dry-run dispatch:

powershell -ExecutionPolicy Bypass -File scripts/queue_windows_python_publish_workflow.ps1 `
  -WorkflowFile .github/workflows/cf_basic_sinks_windows_publish.yml `
  -PackageDir sandcastle/cf_basic_steps/cf_basic_sinks `
  -PublishTarget testpypi `
  -Ref main `
  -RequireLocalPass `
  -DryRun