Skip to main content

OSS Implementation of a DataJunction Reflection Service

Project description

DJ Reflection Service

The reflection service polls the DJ core service for all nodes with associated tables, whether source tables or materialized tables. For each node, it refreshes the node's schema based on the associated table's schema that it retrieves from the query service. It also retrieves the available partitions and the valid through timestamp of these tables and reflects them accordingly to DJ core.

This service uses a celery beat scheduler, with a configurable polling interval that defaults to once per hour and async tasks for each node's reflection.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

datajunction_reflection-0.0.98.tar.gz (87.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

datajunction_reflection-0.0.98-py3-none-any.whl (5.4 kB view details)

Uploaded Python 3

File details

Details for the file datajunction_reflection-0.0.98.tar.gz.

File metadata

  • Download URL: datajunction_reflection-0.0.98.tar.gz
  • Upload date:
  • Size: 87.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for datajunction_reflection-0.0.98.tar.gz
Algorithm Hash digest
SHA256 393561cb3ab730da75cdc5966496adaf14006a8512df331f9c618e198b048d66
MD5 0f3e79a6be8e1cac9fa6df7f61be72f0
BLAKE2b-256 4cca60d8ef0a5859e7c86ec88fa667f1892b4791ea2a8be9672a981be6858e33

See more details on using hashes here.

Provenance

The following attestation bundles were made for datajunction_reflection-0.0.98.tar.gz:

Publisher: publish.yml on DataJunction/dj

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file datajunction_reflection-0.0.98-py3-none-any.whl.

File metadata

File hashes

Hashes for datajunction_reflection-0.0.98-py3-none-any.whl
Algorithm Hash digest
SHA256 72f6ac4a9aca518095c83bae193c743424dcea8534a97729a9ccd68af13b4793
MD5 31cfb5f515b0d5dbc9976c5ce92b59a0
BLAKE2b-256 956eff6fa655bd71890aba869e526122b5107c9b1d6fe7cfa8c10bdeb92cbf95

See more details on using hashes here.

Provenance

The following attestation bundles were made for datajunction_reflection-0.0.98-py3-none-any.whl:

Publisher: publish.yml on DataJunction/dj

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page