Skip to main content

OSS Implementation of a DataJunction Reflection Service

Project description

DJ Reflection Service

The reflection service polls the DJ core service for all nodes with associated tables, whether source tables or materialized tables. For each node, it refreshes the node's schema based on the associated table's schema that it retrieves from the query service. It also retrieves the available partitions and the valid through timestamp of these tables and reflects them accordingly to DJ core.

This service uses a celery beat scheduler, with a configurable polling interval that defaults to once per hour and async tasks for each node's reflection.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

datajunction_reflection-0.0.136.tar.gz (87.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

datajunction_reflection-0.0.136-py3-none-any.whl (5.4 kB view details)

Uploaded Python 3

File details

Details for the file datajunction_reflection-0.0.136.tar.gz.

File metadata

File hashes

Hashes for datajunction_reflection-0.0.136.tar.gz
Algorithm Hash digest
SHA256 4f658e436dac74e7db5e0ad2b39070f9cecd95a3c5cf8859a7069a3bc8aad322
MD5 77b09c67699bf4a838c7caced0425f85
BLAKE2b-256 64c1797b936a1ecd067ba749a1697da2d2b256dbabc4af591214b5e6a2e26aee

See more details on using hashes here.

Provenance

The following attestation bundles were made for datajunction_reflection-0.0.136.tar.gz:

Publisher: publish.yml on DataJunction/dj

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file datajunction_reflection-0.0.136-py3-none-any.whl.

File metadata

File hashes

Hashes for datajunction_reflection-0.0.136-py3-none-any.whl
Algorithm Hash digest
SHA256 cf5a0a30c4466ff918e3ab405e11c8d65556b64fdc4ab457435300466c29e872
MD5 486300bd0cf43192516b8170f6641cfe
BLAKE2b-256 34b8b9d80592e23c9c17651ac4c6c7905ee4f9802a86e6a1913b9c6715a00f07

See more details on using hashes here.

Provenance

The following attestation bundles were made for datajunction_reflection-0.0.136-py3-none-any.whl:

Publisher: publish.yml on DataJunction/dj

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page