OData implementation

This drb-impl-odata module implements the OData protocol access following the Copernicus Space Component schema with DRB data model. It is able to navigate among Product entities of a OData service.

Nodes

ODataServiceNode

Represents the OData service. This node has no attribute and has as children Product entities of the service defined by ProductNode. A specific ProductNode can be retrieved during the bracket and slash navigation by his Name(str) or by his Id(UUID).

OData services required can be access using the second (optional) parameter auth of ODataServiceNode, this parameter must be an requests.auth.AuthBase object, see requests documentation for more details.

ODataProductNode

Represents a Product entity of the OData service. This node has as attribute properties of the associated entity and has for unique child a ProductAttributeNode

ODataProductAttributeNode

This node allowing to represent the navigation link between the Product entity and its attributes. It has no attribute and has as children Attribute entities associated to the Product entity, defined by AttributeNode

ODataAttributeNode

Represents an Attribute entity. This node has no child and has as attribute properties associated to the Attribute entity.

Predicate

ODataCustomQuery

This predicate allows to retrieve a specific subset of children of an ODataServiceNode.

Cache set up

Some cache has been implemented to limit the number of requests to the server. These cache evicts cache entries based on both time and space. the time and the cache size can be changed by adding these two variables to the runtime environment.

DRB_ODATA_NODE_REQUEST_CACHE_EXPIRE_TIME_SEC = 120
DRB_ODATA_NODE_REQUEST_CACHE_MAX_ELEMENTS = 32

Here are all the queries that use a cache

def req_svc(odata: OdataNode) -> dict:
def req_svc_products(odata: OdataNode, **kwargs) -> list:
def req_product_by_uuid(odata: OdataNode, prd_uuid: str) -> dict:
def req_product_attributes(odata: OdataNode, prd_uuid: str) -> List[dict]:

The time of the cache eviction can be change by calling reset_cache_expiration_time(sec=1) :

req_svc_products.reset_expiration_time(sec=1)

Installation

pip install drb-impl-odata

Examples

from uuid import UUID
from requests.auth import HTTPBasicAuth
from drb_impl_odata import ODataServiceNode, \
    ODataProductNode, ODataAttributeNode, ODataQueryPredicate

# generate ODataServiceNode without authentication
odata = ODataServiceNode('https://my.csc.odata.com')
# generate ODataServiceNode with authentication mechanism
odata = ODataServiceNode('https://my.csc.odata.com',
                         auth=HTTPBasicAuth('usr', 'pwd'))

# total number of children
product_count = len(odata)

# retrieve first ODataProductNode
node_idx = odata[0]

# retrieve last ODataProductNode
node_idx = odata[-1]

# retrieve 10 first products
products = odata.children[:10]

# retrieve Product by name
name = 'S2B_OPER_MSI_L0__GR_EPAE_..._D05_N02.06.tar'
node_name_list = odata[name]  # returns a list
node_name = odata[name]  # returns first occurrence of the list

# retrieve Product by UUID
uuid = UUID('0723d9bf-02a2-3e99-b1b3-f6d81de84b62')
node_uuid = odata[uuid]


# get product attributes
prd_node = odata[uuid]
attr_node = prd_node['Attributes']['Footprint']

attr_type = attr_node.get_attribute('ValueType')
attr_value = attr_node.value

# filter and order products
filtered_children = odata / ODataQueryPredicate(filter="startswith(Name,'S1')",
                                                order="ContentLength desc")

The same example with DHus catalog (https://scihub.copernicus.eu/dhus/odata/v2)

The only change is

# get product attributes
# For DhuS ValueType not exist in Attributes...
attr_type = attr_node.get_attribute('ContentType')

A similar example with ONDA-DIAS catalog (https://catalogue.onda-dias.eu/dias-catalogue)

import uuid
from drb_impl_odata import ODataServiceNode, ODataQueryPredicate

svc_url = 'https://catalogue.onda-dias.eu/dias-catalogue'

# generate ODataServiceNode without authentication
odata = ODataServiceNode(svc_url)

# retrieve Product by UUID
uuid_node = uuid.UUID('34a0a4ed-0246-4a57-827d-70350b96d03d')
node_uuid = odata[uuid_node]

attr_node = node_uuid['Attributes']
children = attr_node.children
foot_print = node_uuid.get_attribute('footprint')
print(foot_print)
print(attr_node['Online quality check'].value)

# search by product type limited two the 2 first result
filtered_children = odata / ODataQueryPredicate(
            search='"(platformName:Sentinel-2) AND (productType:S2MSI2A)"',
            top="2")