Skip to main content

OSS Implementation of a DataJunction Reflection Service

Project description

DJ Reflection Service

The reflection service polls the DJ core service for all nodes with associated tables, whether source tables or materialized tables. For each node, it refreshes the node's schema based on the associated table's schema that it retrieves from the query service. It also retrieves the available partitions and the valid through timestamp of these tables and reflects them accordingly to DJ core.

This service uses a celery beat scheduler, with a configurable polling interval that defaults to once per hour and async tasks for each node's reflection.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

datajunction_reflection-0.0.140.tar.gz (87.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

datajunction_reflection-0.0.140-py3-none-any.whl (5.4 kB view details)

Uploaded Python 3

File details

Details for the file datajunction_reflection-0.0.140.tar.gz.

File metadata

File hashes

Hashes for datajunction_reflection-0.0.140.tar.gz
Algorithm Hash digest
SHA256 92cbf6a9767dd5e2076532514a9e35c4d9be5f79f8d02391190d010c0e2f4b79
MD5 d75e6bc5149a74bd8857e10ab49b1c5e
BLAKE2b-256 893718ecb07ab87db99efe275e3c494716f9b31674c8021925ee706946ed06d9

See more details on using hashes here.

Provenance

The following attestation bundles were made for datajunction_reflection-0.0.140.tar.gz:

Publisher: publish.yml on DataJunction/dj

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file datajunction_reflection-0.0.140-py3-none-any.whl.

File metadata

File hashes

Hashes for datajunction_reflection-0.0.140-py3-none-any.whl
Algorithm Hash digest
SHA256 69a3d1107381053bb2f9e220a1cd8ff1469d1f540d7e817761d3b7ec663a0238
MD5 e0e0c6cfda62ab4e846d822770de90d4
BLAKE2b-256 96a7a801bff4f5e6b71b4b5e3a1da524cf4b933d640e254bc449cbc4e924a7a5

See more details on using hashes here.

Provenance

The following attestation bundles were made for datajunction_reflection-0.0.140-py3-none-any.whl:

Publisher: publish.yml on DataJunction/dj

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page