Skip to main content

OSS Implementation of a DataJunction Reflection Service

Project description

DJ Reflection Service

The reflection service polls the DJ core service for all nodes with associated tables, whether source tables or materialized tables. For each node, it refreshes the node's schema based on the associated table's schema that it retrieves from the query service. It also retrieves the available partitions and the valid through timestamp of these tables and reflects them accordingly to DJ core.

This service uses a celery beat scheduler, with a configurable polling interval that defaults to once per hour and async tasks for each node's reflection.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

datajunction_reflection-0.0.123.tar.gz (87.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

datajunction_reflection-0.0.123-py3-none-any.whl (5.4 kB view details)

Uploaded Python 3

File details

Details for the file datajunction_reflection-0.0.123.tar.gz.

File metadata

File hashes

Hashes for datajunction_reflection-0.0.123.tar.gz
Algorithm Hash digest
SHA256 d4db91a3d0832935ec93ed2492fa70142d1cd8a8c27ed53d8d033221b40ac2e1
MD5 3064b14681c867cb640942632f84896f
BLAKE2b-256 bbcc4c2f903a084d514e50cac858aecc7b6da5e3fa0bbaeee81def084445f0e5

See more details on using hashes here.

Provenance

The following attestation bundles were made for datajunction_reflection-0.0.123.tar.gz:

Publisher: publish.yml on DataJunction/dj

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file datajunction_reflection-0.0.123-py3-none-any.whl.

File metadata

File hashes

Hashes for datajunction_reflection-0.0.123-py3-none-any.whl
Algorithm Hash digest
SHA256 2e6cbdae84502b2f3343a0e32991afcdec48b22c8498a811e43c9448346d0dd6
MD5 8b2fcd85a5d0018d49be588a243d9e94
BLAKE2b-256 ddddaaab286ad414b9d67862061d61557f77236ec9bb59a4f7717a37d56420ed

See more details on using hashes here.

Provenance

The following attestation bundles were made for datajunction_reflection-0.0.123-py3-none-any.whl:

Publisher: publish.yml on DataJunction/dj

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page