Skip to main content

OSS Implementation of a DataJunction Reflection Service

Project description

DJ Reflection Service

The reflection service polls the DJ core service for all nodes with associated tables, whether source tables or materialized tables. For each node, it refreshes the node's schema based on the associated table's schema that it retrieves from the query service. It also retrieves the available partitions and the valid through timestamp of these tables and reflects them accordingly to DJ core.

This service uses a celery beat scheduler, with a configurable polling interval that defaults to once per hour and async tasks for each node's reflection.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

datajunction_reflection-0.0.135.tar.gz (87.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

datajunction_reflection-0.0.135-py3-none-any.whl (5.4 kB view details)

Uploaded Python 3

File details

Details for the file datajunction_reflection-0.0.135.tar.gz.

File metadata

File hashes

Hashes for datajunction_reflection-0.0.135.tar.gz
Algorithm Hash digest
SHA256 0625d96e0113017df09329f9747bd9a41ca768359385de816f6e6bb751c73bd1
MD5 dd7fae402ea0ababaa2097265ab65a95
BLAKE2b-256 ee255718021644e084ec4708fe14cf697639d5c3a27dbb549c1d098f02b8e1dd

See more details on using hashes here.

Provenance

The following attestation bundles were made for datajunction_reflection-0.0.135.tar.gz:

Publisher: publish.yml on DataJunction/dj

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file datajunction_reflection-0.0.135-py3-none-any.whl.

File metadata

File hashes

Hashes for datajunction_reflection-0.0.135-py3-none-any.whl
Algorithm Hash digest
SHA256 e051d695f984bf6b5db899339c57a77eaff6dbb93ee7cf0e8e4c15ee791434a1
MD5 56c5c3366b7d5bf7142c9bdcf8ba3e38
BLAKE2b-256 8001b87a33f718a26ac483f258c593c694c7d5f0ada57e21b95af04d0c76de22

See more details on using hashes here.

Provenance

The following attestation bundles were made for datajunction_reflection-0.0.135-py3-none-any.whl:

Publisher: publish.yml on DataJunction/dj

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page