datahub-api-connector 1.3

This package simplifies the calls to Data Hub's API.

No magic. You need to follow the Data Hub Swagger Documentation for correct formatting of your requests

Be aware that this is a new package, certainly missing many features. Feel free to contribute.

You first need to create an instance of the ApiConnector class with following parameters:

ATTENTION: this package can only be used from Data Hub version 7.0 (July 1, 2025) onward, as it uses the authentication to the the new technology (Keycloak). For previous versions, the opinum-api-connector package must be used (https://github.com/opinum/opinum-api-connector) instead.

VERSION 1.3 Sturdiness and correctness release.

• The token lifetime now follows the real expiry returned by the authentication server (with a 120 seconds safety margin) instead of a fixed 3 minutes limit, and is renewed with its refresh token when possible.
• All calls now go through a single persistent connection pool (see the new pool_size parameter) instead of opening a new connection each time, which exhausted the available sockets on long multi-threaded runs.
• Fixed a retried call sending a doubly encoded body (rejected with a 400), and a token renewed between two attempts not being used. Read timeouts are now retried as well, and the last failed attempt no longer waits before raising.
• Fixed operation_timeout_sec being ignored on push_data when operation_id was given too, and file names containing special characters on send_file_to_storage.
• multi_thread_request_on_path now really uses threads (it raised a pickling error before) and splits the calls on max_parameter_entities as documented, instead of on max_futures. Expect different call sizes than in 1.2.
• ApiConnector can be closed, and used as a context manager, to release its connections.

VERSION 1.2 Fixed operation_id on PUSH calls, and added operation_timeout_sec. Also added the possibility to request a count of items on some calls with the IncludeItemsCount optional parameter.

VERSION 1.1 Improved sturdiness. Added thread lock on token requests, and a default timeout of 10 seconds on all requests (incliding token)

environment

a dictionary of environment variables

if None, ApiConnector uses your environment variables (os.environ)

Mandatory environment variables are:

• DATAHUB_USERNAME: the Data Hub user.
  TAKE CARE: if this user has access to multiple tenants and if you do not specify a tenant id, ApiConnector will use the last tenant used.
• DATAHUB_PASSWORD: the password for the user
• DATAHUB_CLIENT_ID: the client id for accessing the API
• DATAHUB_CLIENT_SECRET the corresponding secret

Optional environment variables are:

• DATAHUB_API_URL: another API URL than the Europe SaaS one (https://api.opinum.com)
• DATAHUB_AUTH_URL: another authentication URL than the Europe SaaS one (https://auth.opinum.com)
• DATAHUB_PUSH_URL: another push URL than the Europe SaaS one (https://push.opinum.com)
• DATAHUB_SCOPE: the scope of you session (default: "datahub-api")
  if you want to push data, the scope should be "datahub-api push-data"

account_id

one of the tenant ids available for the Data Hub user (default: None)

retries_when_connection_failure

number of extra attempts when no 200 or 204 return code (default: 0, maximum: 5)

request_timeout

timeout value in seconds on all requests (including fetch token) (default: 10)

log_level

sets log level for the module (default: INFO)

pool_size

size of the connection pool shared by all calls of the instance (default: 32)

it must be at least as large as the number of threads sharing the instance (the workers parameter of multi_thread_request_on_path, default: 16)

Once you have your ApiConnector instance, you may use the class methods

• get
• post
• patch
• put
• delete
• send_file_to_storage

All keyword arguments will be converted to path parameters in the API call with one important exception, the data keyword referring to the body of your call.

There are two other class methods for data pushing because we have another API for this

• push_data
• push_dataframe_data

There is a little bit of magic with the method multi_thread_request_on_path that splits a list of parameters Allowing to make parallel calls.