Skip to main content

File output destination connector package for RETL State/Event contracts.

Project description

RETL File Destination

condor-retl-file is a first-party destination connector that writes RETL State Operations and Event imports to local CSV export drops. It exposes the retl/file connector through the retl.destinations entry point group.

Surfaces

  • csv_state_operations accepts State Operations, supports upsert and remove, and writes upserts.csv and removes.csv when those operation kinds are present.
  • csv_event_imports accepts Event imports and writes imports.csv.

Both surfaces accept RETL's common identifier types without requiring a specific identifier, because the connector writes identifier values through to the CSV rather than translating them for a partner API.

Each non-dry-run submission creates a new timestamped export directory under output_dir and writes a manifest.json with file names, row counts, byte sizes, SHA-256 checksums, RETL batch ids, and operation counts. Successful file writes produce definitive succeeded delivery evidence.

Binding Config

The file connector does not use secrets or credentials.

  • output_dir is required and points to the parent directory for export drops.
  • file_batch_max_rows controls RETL file batch planning. It defaults to 10000.
  • create_parent_dirs controls whether missing output parents are created during non-dry-run submission. It defaults to true.

output_dir, file_batch_max_rows, and create_parent_dirs are namespace-loadable public config:

destination = retl.destinations.load(
    "retl/file",
    binding_name="file_exports",
    config_namespace="destinations.file",
)

CSV Shape

CSV files use stable structured columns:

  • operation
  • record_identity
  • identifiers_json
  • key_json
  • payload_json
  • target
  • occurred_at
  • collect_id
  • sequence_order
  • payload_fingerprint

V1 writes new export drops for every submission. It does not mutate existing CSV files and does not claim exactly-once file delivery; downstream consumers should use manifest batch ids and checksums for deduplication.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

condor_retl_file-0.1.0.tar.gz (12.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

condor_retl_file-0.1.0-py3-none-any.whl (12.7 kB view details)

Uploaded Python 3

File details

Details for the file condor_retl_file-0.1.0.tar.gz.

File metadata

  • Download URL: condor_retl_file-0.1.0.tar.gz
  • Upload date:
  • Size: 12.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for condor_retl_file-0.1.0.tar.gz
Algorithm Hash digest
SHA256 8bbd5d0deff6abf6a0708401bd37e72692d04416f8ca4503e600a0d7f4028129
MD5 8c5cf70686f700aa20c5837fd98dca68
BLAKE2b-256 98c43606853e6b9adc8b8445c37046b30064b7f074a3e0b243adfb9e54aa55ba

See more details on using hashes here.

Provenance

The following attestation bundles were made for condor_retl_file-0.1.0.tar.gz:

Publisher: publish.yml on condorgraph/condor-retl

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file condor_retl_file-0.1.0-py3-none-any.whl.

File metadata

File hashes

Hashes for condor_retl_file-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 15ebd2d2f505e5da48585e52e6a927e7eb43bc24bfb0a25ec584d0f257aafebc
MD5 54419abb98ce0269a74c5a0c89f07436
BLAKE2b-256 d81a199ddbb444e2331dcb29278fe306fd5f280043b91e12dedb69998d2cee9d

See more details on using hashes here.

Provenance

The following attestation bundles were made for condor_retl_file-0.1.0-py3-none-any.whl:

Publisher: publish.yml on condorgraph/condor-retl

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page