Skip to main content

OSS Implementation of a DataJunction Reflection Service

Project description

DJ Reflection Service

The reflection service polls the DJ core service for all nodes with associated tables, whether source tables or materialized tables. For each node, it refreshes the node's schema based on the associated table's schema that it retrieves from the query service. It also retrieves the available partitions and the valid through timestamp of these tables and reflects them accordingly to DJ core.

This service uses a celery beat scheduler, with a configurable polling interval that defaults to once per hour and async tasks for each node's reflection.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

datajunction_reflection-0.0.66.tar.gz (87.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

datajunction_reflection-0.0.66-py3-none-any.whl (5.4 kB view details)

Uploaded Python 3

File details

Details for the file datajunction_reflection-0.0.66.tar.gz.

File metadata

  • Download URL: datajunction_reflection-0.0.66.tar.gz
  • Upload date:
  • Size: 87.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for datajunction_reflection-0.0.66.tar.gz
Algorithm Hash digest
SHA256 465d329b88564238c84d58eeb9e44c7daa9a26d09eaa229194c38af23f09efed
MD5 29e86d2b8cab43387ebc4cd8f5b4f448
BLAKE2b-256 90cfda2936910f6349c92914451994064f0c31c87b90ea226c2e9a1c06988006

See more details on using hashes here.

Provenance

The following attestation bundles were made for datajunction_reflection-0.0.66.tar.gz:

Publisher: publish.yml on DataJunction/dj

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file datajunction_reflection-0.0.66-py3-none-any.whl.

File metadata

File hashes

Hashes for datajunction_reflection-0.0.66-py3-none-any.whl
Algorithm Hash digest
SHA256 cc8960bf3944a9c72c8683f03acd939977d9f515554eb9dadb36d32c5d928863
MD5 55739adb10b3c98367df8913400d1483
BLAKE2b-256 3f892a90a2689eae2bd07dfe67bb751b7304a090cecebfa9c899c1ca9247daf0

See more details on using hashes here.

Provenance

The following attestation bundles were made for datajunction_reflection-0.0.66-py3-none-any.whl:

Publisher: publish.yml on DataJunction/dj

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page