Skip to main content

Utilities for easier development of extractors for CDF

Project description

Cognite logo

Cognite Python extractor-utils

Build Status Documentation Status codecov PyPI version PyPI - Python Version License Code style: black

The extractor-utils package is an extension of the Cognite Python SDK intended to simplify the development of data extractors or other integrations for Cognite Data Fusion.

Documentation is hosted here, including a quickstart tutorial.

The changelog is found here.

Overview

The best way to start a new extractor project is to use the cogex CLI. You can install that from PyPI:

pip install cognite-extractor-manager

To initialize a new extractor project, run

cogex init

in the directory you want your extractor project in. The cogex CLI will prompt you for some information about your project, and then set up a poetry environment, git repository, commit hooks with type and style checks and load a template.

Extensions

Some source systems have a lot in common, such as RESTful APIs or systems exposing as MQTT. We therefore have extensions to extractor-utils tailroed to these protocols. These can be found in separate packages:

Contributing

The package is open source under the Apache 2.0 license, and contribtuions are welcome.

This project adheres to the Contributor Covenant v2.0 as a code of conduct.

Development environment

We use poetry to manage dependencies and to administrate virtual environments. To develop extractor-utils, follow the following steps to set up your local environment:

  1. Install poetry if you haven't already.

  2. Clone repository:

    $ git clone git@github.com:cognitedata/python-extractor-utils.git
    
  3. Move into the newly created local repository:

    $ cd python-extractor-utils
    
  4. Create virtual environment and install dependencies:

    $ poetry install
    

Code requirements

All code must pass black and isort style checks to be merged. It is recommended to install pre-commit hooks to ensure this locally before commiting code:

$ poetry run pre-commit install

Each public method, class and module should have docstrings. Docstrings are written in the Google style. Please include unit and/or integration tests for submitted code, and remember to update the changelog.

Project details


Release history Release notifications | RSS feed

This version

3.1.4

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cognite_extractor_utils-3.1.4.tar.gz (41.0 kB view details)

Uploaded Source

Built Distribution

cognite_extractor_utils-3.1.4-py3-none-any.whl (50.0 kB view details)

Uploaded Python 3

File details

Details for the file cognite_extractor_utils-3.1.4.tar.gz.

File metadata

  • Download URL: cognite_extractor_utils-3.1.4.tar.gz
  • Upload date:
  • Size: 41.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.9.2 readme-renderer/37.3 requests/2.28.1 requests-toolbelt/0.10.1 urllib3/1.26.13 tqdm/4.64.1 importlib-metadata/6.0.0 keyring/23.13.1 rfc3986/2.0.0 colorama/0.4.6 CPython/3.8.15

File hashes

Hashes for cognite_extractor_utils-3.1.4.tar.gz
Algorithm Hash digest
SHA256 76027fb549e9b190d5a0b0a1ab8ed16bc376e003d1e16e8a4e98710b6e850c12
MD5 0b4cf04155f486e07e6f6623eadddb54
BLAKE2b-256 5dc01b737f934eb4738d7a50743e4f4e0fc947090f42f277dd4a1e207f297ba9

See more details on using hashes here.

File details

Details for the file cognite_extractor_utils-3.1.4-py3-none-any.whl.

File metadata

  • Download URL: cognite_extractor_utils-3.1.4-py3-none-any.whl
  • Upload date:
  • Size: 50.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.9.2 readme-renderer/37.3 requests/2.28.1 requests-toolbelt/0.10.1 urllib3/1.26.13 tqdm/4.64.1 importlib-metadata/6.0.0 keyring/23.13.1 rfc3986/2.0.0 colorama/0.4.6 CPython/3.8.15

File hashes

Hashes for cognite_extractor_utils-3.1.4-py3-none-any.whl
Algorithm Hash digest
SHA256 c0f4dea87d74ad6cfa38459e15cbee2416ea38ae19b06a47de6be2bfef4543c9
MD5 eb42d9f1169c8475487fa58ad4d38686
BLAKE2b-256 e566d66087e7881e5f93cd08654e150f0f0d12bce2a0e3e30fa9a1ffe565eba0

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page