odd-ge-adapter

ODD GreatExpectation adapter

Library for GreatExpectation schema compatible json files for suites and validation, reads them from s3 storage and ingests to OpenDataDiscovery platform.

Suites must include special metadata about folder with data which will be validated in the future. Example

{
  "expectation_suite_name": "notification",
  ...
  "meta": {
    "batch_kwargs": {
      ...
      "odd_metadata": [
        "s3://bucket/folder"
      ]
    }
  }
}

Validation results must include metadata about which file was validated. Example

{
  "meta": {
    "batch_kwargs": {
      ...
      "odd_metadata": [
        "s3://bucket/folder/file.csv"
      ]
    },
    ...
    "validation_time": "20220525T174015.717924Z"
  },
  "results": [...],
  "success": true
}

Library usage

Install package

pip install odd-ge-adapter

Example:

from odd_models.api_client import ODDApiClient
from odd_ge_adapter.domain.plugin import S3StoragePlugin
from odd_ge_adapter.adapters.s3.adapter import Adapter

platform = ODDApiClient(<PLATFORM_URL>)

# aws credentials can be omitted and taken from the ENV variables 
config = S3StoragePlugin(
    type="s3",
    name="s3_storage",
    bucket="",
    results_key="",
    suites_key="",
    # aws_account_id="",
    # aws_region="",
    # aws_access_key_id="",
    # aws_secret_access_key="",
    # aws_session_token=""
)

adapter = Adapter(config)
platform.post_data_entity_list(data=adapter.get_data_entity_list())

As collector usage

In progress...

Metadata

Release files for odd-ge-adapter 0.2.13

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for odd-ge-adapter 0.2.13
File	Size	Uploaded	Provenance
odd-ge-adapter-0.2.13.tar.gz	12.1 kB	Jun 27, 2022	Without provenance	Details

Built distribution (wheel)

Table of built distributions (wheels) for odd-ge-adapter 0.2.13
File	Interpreter	ABI	Platform	Reset
odd_ge_adapter-0.2.13-py3-none-any.whl 16.2 kB Jun 27, 2022 Without provenance	Python 3	none	any	Details

Total release size: 28.4 kB

Release files / odd-ge-adapter-0.2.13.tar.gz

Download URL	odd-ge-adapter-0.2.13.tar.gz
Size	12.1 kB
Tags	Source
SHA-256 checksum How to use checksums	`443e8d8759db538cf7959fc1c5d317a15335c42c85d9a773a7bc09406195dc1e`
BLAKE2b-256 checksum How to use checksums	`a2bbda98271a9b8d70c156eeffa7629880c0a9d075bf4a05b3f1b52edb19da8f`
Upload date	Jun 27, 2022
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/1.1.13 CPython/3.7.12 Darwin/21.1.0`

Release files / odd_ge_adapter-0.2.13-py3-none-any.whl

Download URL	odd_ge_adapter-0.2.13-py3-none-any.whl
Size	16.2 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`8130210abcaf2a684737f9575da03b57c8026bdaac4b0874b213327f5322930c`
BLAKE2b-256 checksum How to use checksums	`f8687a6402a0725e02ba79ab6a8949e2042cf2bc2a1338f28c4f1a794146ffc8`
Upload date	Jun 27, 2022
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/1.1.13 CPython/3.7.12 Darwin/21.1.0`

Security for odd-ge-adapter 0.2.13

Report project as malware

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.