Skip to main content

Python SDK for Osiris (Energinet DataPlatform).

Project description

Pylint Bandit Flake8 Mypy Pytest

osiris-sdk

Installing

$ pip install osiris-sdk

The SDK requires Python 3.

Getting Started

To get started with the SDK you will need the URL to the Osiris-ingress API and the tenant ID for the organisation who runs the API. Furthermore, you will need to register your application withing the tenant using Azure App Registration. You will also need to create a dataset in the DataPlatform.

Data application registration

An App Registration with credentials are required to upload data to the DataPlatform through the Osiris Ingress API.

Prerequisites

  • The dataset has been created through the Data Platform.
  • The Azure CLI is installed on your workstation

Steps

Login with the Azure CLI with the following command:

az login

You can also specify a username and password with:

az login -u <username> -p <password>

Create an App Registration

The App Registration serves as a registration of trust for your application (or data publishing service) towards the Microsoft Identity Platform (allowing authentication).

This is the "identity" of your application. Note that an App Registration is globally unique.

Run the following command:

az ad app create --display-name "<YOUR APP NAME>"

The application name should be descriptive correlate to the application/service you intend to upload data with.

Take note of the appId GUID in the returned object.

Create a Service Principal and credentials

The Service Principal and credentials are what enables authorization to the Data Platform.

Create a Service Principal using the appId GUID from when creating the App Registration:

az ad sp create --id "<appID>"

Then create a credential for the App Registration:

az ad app credential reset --id "<appID>"

NOTE: Save the output somewhere secure. The credentials you receive are required to authenticate with the Osiris Ingress API.

Grant access to the dataset

The application must be granted read- and write-access to the dataset on the Data Platform.

Add the application you created earlier, using the <YOUR APP NAME> name, to the read- and write-access lists.

Usage

Here are some simple examples on how to use the SDK.

Upload

The following is a simple example which shows how you can upload files using the Osiris SDK:

from osiris.apis.ingress import Ingress
from osiris.core.azure_client_authorization import ClientAuthorization


client_auth = ClientAuthorization(tenant_id=<TENANT_ID>,
                                  client_id=<CLIENT_ID>,
                                  client_secret=<CLIENT_SECRET>)

ingress = Ingress(client_auth=client_auth,
                  ingress_url=<INGRESS_URL>,
                  dataset_guid=<DATASET_GUID>)

file = open('test_file.json', 'rb')

# Without schema validation and a JSON file
ingress.upload_json_file(file, False)

# With schema validation and a JSON file
ingress.upload_json_file(file, True)

# With schema validation, a JSON file and event time
ingress.upload_json_file_event_time(file, '2021-01-01', True)

# Arbitrary file
ingress.upload_file(file)

# Save state file
with open('state.json', 'r') as state:
    ingress.save_state(state)

# Retrieve state
state = ingress.retrieve_state()

Download

The following is a simple example which shows how you can download files using the Osiris SDK:

from osiris.apis.egress import Egress
from osiris.core.azure_client_authorization import ClientAuthorization


client_auth = ClientAuthorization(tenant_id=<TENANT_ID>,
                                  client_id=<CLIENT_ID>,
                                  client_secret=<CLIENT_SECRET>)

egress = Egress(client_auth=client_auth,
                egress_url=<EGRESS_URL>,
                dataset_guid=<DATASET_GUID>)

# JSON file
content_json = egress.download_json_file('2021-01-01', '2021-01-03')

The full list of methods available in Egress can be found in the source code.

Ingress Adapter

The following is a simple example which shows how you can create a new ingress adapter.

import json
from osiris.adapters.ingress_adapter import IngressAdapter


class MyAdapter(IngressAdapter):
    def retrieve_data(self) -> bytes:
        return json.dumps('Hello World').encode('UTF-8')


def main():
    client_auth = ClientAuthorization(tenant_id=<TENANT_ID>,
                                      client_id=<CLIENT_ID>,
                                      client_secret=<CLIENT_SECRET>)

    adapter = MyAdapter(client_auth=client_auth,
                        ingress_url=<INGRESS_URL>,
                        dataset_guid=<DATASET_GUID>)

    # as json data
    adapter.upload_json_data(schema_validate=False)

    # or as arbitrary data
    adapter.upload_data()

    # or json data with a path corresponding to the given event time
    adapter.upload_json_data_event_time(schema_validate=False)

    # or data with a path corresponding to the given event time
    adapter.upload_data_event_time()

if __name__ == '__main__':
    main()

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

osiris-sdk-1.0.13.tar.gz (21.6 kB view details)

Uploaded Source

Built Distribution

osiris_sdk-1.0.13-py3-none-any.whl (25.0 kB view details)

Uploaded Python 3

File details

Details for the file osiris-sdk-1.0.13.tar.gz.

File metadata

  • Download URL: osiris-sdk-1.0.13.tar.gz
  • Upload date:
  • Size: 21.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.1 CPython/3.8.9

File hashes

Hashes for osiris-sdk-1.0.13.tar.gz
Algorithm Hash digest
SHA256 150672421f259a4bb148546a94949156b22ede5a0a4a1756e52623e4662f17fe
MD5 e2f9181af368010e50dff9e0246ae4c1
BLAKE2b-256 0f225c11857b7742d327252a4fe1f51433a6d7938c0df240a25628bd49e9cd1c

See more details on using hashes here.

File details

Details for the file osiris_sdk-1.0.13-py3-none-any.whl.

File metadata

  • Download URL: osiris_sdk-1.0.13-py3-none-any.whl
  • Upload date:
  • Size: 25.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.1 CPython/3.8.9

File hashes

Hashes for osiris_sdk-1.0.13-py3-none-any.whl
Algorithm Hash digest
SHA256 df3680bdb03104800a55a7802272f58d46963b4866b235676a226f6dd14acd7d
MD5 c46b148d22b1fb505a4c9fbb660ceeee
BLAKE2b-256 3b0c5ff2b369f21ca9389fe60ecb2ed2cfde673d20e9a9152cb0044eb84451bd

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page