Skip to main content

OSS Implementation of a DataJunction Reflection Service

Project description

DJ Reflection Service

The reflection service polls the DJ core service for all nodes with associated tables, whether source tables or materialized tables. For each node, it refreshes the node's schema based on the associated table's schema that it retrieves from the query service. It also retrieves the available partitions and the valid through timestamp of these tables and reflects them accordingly to DJ core.

This service uses a celery beat scheduler, with a configurable polling interval that defaults to once per hour and async tasks for each node's reflection.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

datajunction_reflection-0.0.133.tar.gz (87.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

datajunction_reflection-0.0.133-py3-none-any.whl (5.4 kB view details)

Uploaded Python 3

File details

Details for the file datajunction_reflection-0.0.133.tar.gz.

File metadata

File hashes

Hashes for datajunction_reflection-0.0.133.tar.gz
Algorithm Hash digest
SHA256 4c4b3d43e7a57fe6c2bd968adcc145bb27f0a9edf99fc039c541ebcbcf5c97d2
MD5 a3edd4809a09fe309fde868c96bb64f1
BLAKE2b-256 c9876c986bdc6f417a5b6f4649891a30f5674ebd910541e63b1ba66ecd7f65e5

See more details on using hashes here.

Provenance

The following attestation bundles were made for datajunction_reflection-0.0.133.tar.gz:

Publisher: publish.yml on DataJunction/dj

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file datajunction_reflection-0.0.133-py3-none-any.whl.

File metadata

File hashes

Hashes for datajunction_reflection-0.0.133-py3-none-any.whl
Algorithm Hash digest
SHA256 5fee3798e4655f57c84cddfac4be29b8c414a74aa80405795025778c3125ddf1
MD5 ba4a7785a17b70a67ace064b38491d61
BLAKE2b-256 0e1fe72af77e352cd0a5089572814d412db38efc6d8e856266f4cc4ae5e3c3bf

See more details on using hashes here.

Provenance

The following attestation bundles were made for datajunction_reflection-0.0.133-py3-none-any.whl:

Publisher: publish.yml on DataJunction/dj

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page