Skip to main content

drb-iceberg

Purpose: Turn a product topic's iceberg: ontological configuration into a declarative IcebergMapping that describes table name, partitions, column extractors, and derived sub-tables. The add-on performs planning only — no Iceberg catalog I/O.

Contract

IcebergAddon.apply(node, topic=None, **kwargs) returns an IcebergMapping object containing:

  • table: main table name (string)
  • ingestible: ingestion enabled (boolean)
  • ingest_priority: optional priority level (int or None)
  • identifier: unique key extractor (XQuery-based, not executed)
  • partitions: list of partition field definitions
  • columns: column schema with row extractors (delegated to drb-metadata)
  • derived: list of derived (sub-)table definitions

The mapping is declarative: extractors are not executed by the add-on. A separate consumer processes the plan and executes row extraction/table writes.

Dependencies

  • drb>=2.2,<3
  • drb-metadata>=1.4,<2 — provides column extraction via extract_for_class(class_uri, graph)

The iceberg: vocabulary

Attached to a product topic at namespace http://www.gael.fr/iceberg#:

Term Cardinality Type Meaning
ingestible 1 xs:boolean Enables ingestion for this topic.
targetTable 1 string Main table name, e.g. sentinel1.products.
ingestPriority 0..1 xs:int Ingestion priority.
identifier 1 XQuery Unique key of the product row.
partitionStrategy 0..n PartitionStrategy Partition field/transform/order.
derivedTable 0..n DerivedTable Sub-table definition.

PartitionStrategy

  • partitionField (string) — field name
  • partitionTransform (string: identity, year, month, day, etc.)
  • partitionOrder (xs:int) — sort order

DerivedTable

  • tableName (string) — suffix appended to main table with a dot
  • entityListExtractor (XQuery) — yields the list of row nodes
  • entityClass (resource) OR entityMetadataExtractor (XQuery) — column source (entityClass wins if both present)
  • entityIdentifier (XQuery) — unique key per row

Metadata

Release files for drb-iceberg 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for drb-iceberg 0.2.0
File Size Uploaded
drb_iceberg-0.2.0.tar.gz 27.5 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for drb-iceberg 0.2.0
File Interpreter ABI Platform
drb_iceberg-0.2.0-py3-none-any.whl Python 3 none any Details

Total release size: 36.5 kB

Release files / drb_iceberg-0.2.0.tar.gz

Download URL drb_iceberg-0.2.0.tar.gz
Size 27.5 kB
Tags Source
SHA-256 checksum
How to use checksums
59e7cbddc881dfc0b1a7d940b7b7ab01d18178e21f8c00b92490055eeb1b4875
BLAKE2b-256 checksum
How to use checksums
71a745ca99ffc8c7464fc16605a8661a659f380445ce2ddb09e22667b27f2a65
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.11.15

Release files / drb_iceberg-0.2.0-py3-none-any.whl

Download URL drb_iceberg-0.2.0-py3-none-any.whl
Size 8.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
440a8e28420cd56762cc0d201c69bd1fdd1103a3ef55d58dbdee3ce6915f2968
BLAKE2b-256 checksum
How to use checksums
9237cbee9ee47b89afa47f7bba532c5f1fef61411ebd0a30dcf4362288bb76a8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.11.15

Release history Release notifications | RSS feed

This release

0.2.0 This release

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page