cf-basic-sinks 0.1.2

Cogniflow basic sink StepPackage with C++ data hive parquet sink.

cf-basic-sinks

cf-basic-sinks provides native sink steps for pipeline outputs.

Steps

Step	Category	Description
cfsink:DataHiveParquetSinkStep	sink	Write canonical data hive parquet output through the cf_datahive_cpp gatekeeper

Published distribution name:

pip install cf-basic-sinks

Native build prerequisites

cf_basic_sinks is built with scikit-build-core and CMake and requires:

• CPython 3.13
• CMake on PATH
• a Windows C++ toolchain compatible with that CMake installation
• access to cf-pipeline-sdk from the package index
• access to cf-datahive from the package index for the owner-provided native cf_datahive_cpp source surface
• DuckDB amalgamation source (duckdb.cpp) for default static-link builds
• DuckDB native headers/library only when forcing shared-link fallback mode

This package does not publish cf_datahive_cpp separately. The step package consumes the owner-provided native source surface exposed by the installed cf_datahive package. Native data hive write ownership remains under cf_datahive.

Local builds can satisfy the default static-link DuckDB requirement through:

• .native_deps/duckdb/src/duckdb.cpp staged by scripts/setup_native_deps_v2.ps1
• CF_DATAHIVE_CPP_DUCKDB_SOURCE pointing at a duckdb.cpp amalgamation file

Shared-link fallback mode can be selected with CF_DATAHIVE_CPP_DUCKDB_LINKAGE=shared. In that mode, DuckDB inputs are satisfied through either:

• the repo-local .native_deps/duckdb layout created by the setup scripts
• CF_DATAHIVE_CPP_DUCKDB_INCLUDE and CF_DATAHIVE_CPP_DUCKDB_LIB

Publishing

cf_basic_sinks is published with the dedicated Windows workflow:

• Workflow: .github/workflows/cf_basic_sinks_windows_publish.yml
• Package directory: sandcastle/cf_basic_steps/cf_basic_sinks
• PyPI tag: cf-basic-sinks-v<version>
• TestPyPI tag: cf-basic-sinks-v<version>-test

Workflow note:

• test/build jobs provision native deps via scripts/setup_native_deps_v2.ps1
• test/build jobs preinstall published cf-pipeline-sdk and cf-datahive into the job interpreter before invoking the shared publish helper, so the CMake Python subprocesses can discover the owner-provided SDK and data-hive surfaces
• test/build jobs export CF_DATAHIVE_CPP_DUCKDB_LINKAGE=static and CF_DATAHIVE_CPP_DUCKDB_SOURCE from the repo-local .native_deps/duckdb/src layout
• test/build jobs also export CF_DATAHIVE_CPP_DUCKDB_INCLUDE and CF_DATAHIVE_CPP_DUCKDB_LIB to preserve shared-link fallback compatibility

Local preflight:

$env:CF_DATAHIVE_CPP_DUCKDB_LINKAGE = "static"
$env:CF_DATAHIVE_CPP_DUCKDB_SOURCE = (Resolve-Path .native_deps/duckdb/src/duckdb.cpp).Path
$env:CF_DATAHIVE_CPP_DUCKDB_INCLUDE = (Resolve-Path .native_deps/duckdb).Path
$env:CF_DATAHIVE_CPP_DUCKDB_LIB = (Resolve-Path .native_deps/duckdb/lib/duckdb.lib).Path
powershell -ExecutionPolicy Bypass -File scripts/mimic_windows_python_publish_workflow.ps1 `
  -WorkflowFile .github/workflows/cf_basic_sinks_windows_publish.yml `
  -PackageDir sandcastle/cf_basic_steps/cf_basic_sinks `
  -PythonExe py `
  -PythonVersion 3.13

Queue a dry-run dispatch:

powershell -ExecutionPolicy Bypass -File scripts/queue_windows_python_publish_workflow.ps1 `
  -WorkflowFile .github/workflows/cf_basic_sinks_windows_publish.yml `
  -PackageDir sandcastle/cf_basic_steps/cf_basic_sinks `
  -PublishTarget testpypi `
  -Ref main `
  -RequireLocalPass `
  -DryRun