DX API Client Library
Welcome to the DX API Client Library! This library provides a convenient Python interface to interact with the DX API, allowing you to manage datasets, installations, and perform various operations with ease.
Table of Contents
- Features
- Installation
- Prerequisites
- Getting Started
- Usage
- Asynchronous Usage
- Examples
- License
Features
- Authenticate with the DX API using JWT tokens.
- Manage installations and datasets.
- Upload and download data to and from datasets.
- Synchronous and asynchronous support.
- Context managers for handling authentication scopes.
Installation
You can install the library using pip:
pip install mig-dx-api
Prerequisites
- Python 3.10 or higher.
- An application ID (
app_id),a workspace key (workspace_key), and a corresponding private key in PEM format from the Console. - DX API access credentials.
Getting Started
Authentication
The library uses JWT tokens for authentication. You need to provide your app_id, workspace_key, and the path to your private key file when initializing the client.
Initialization
from mig_dx_api import DX
# Initialize the client
dx = DX(app_id='your_app_id', private_key_path='path/to/private_key.pem')
# OR
dx = DX(app_id='your_app_id',private_key='your_private_key')
Alternatively, you can set the environment variables DX_CONFIG_APP_ID, DX_CONFIG_WORKSPACE_KEY, and DX_CONFIG_PRIVATE_KEY_PATH:
export DX_CONFIG_APP_ID='your_app_id'
export DX_CONFIG_WORKSPACE_KEY='your_workspace_key'
export DX_CONFIG_PRIVATE_KEY_PATH='path/to/private_key.pem'
# OR
export DX_CONFIG_PRIVATE_KEY='your_private_key'
And initialize the client without arguments:
dx = DX()
Usage
Who Am I
Retrieve information about the authenticated user:
user_info = dx.whoami()
print(user_info)
Managing Installations
Listing Installations
installations = dx.get_installations()
for installation in installations:
print(installation.name)
Accessing an Installation Context
Use the installation context to perform operations related to a specific installation:
# Find an installation by name or ID
installation = dx.installations.find(install_id=1)
# Get an installation by workspace_id
installation = dx.get_installations(workspace_id=1)[0]
if DX_CONFIG_WORKSPACE_KEY is set you don't need to pass in the workspace_key as a param on dx.installation
Use the installation context
with dx.installation(installation, workspace_key=workspace_key) as ctx:
# Perform operations within the context
datasets = list(ctx.datasets)
for dataset in datasets:
print(dataset.name)
Or enter context with a lookup by name:
with dx.installation(install_id=1) as ctx:
# Perform operations within the context
datasets = list(ctx.datasets)
for dataset in datasets:
print(dataset.name)
Or enter context with a lookup by workspace_id:
with dx.installation(workspace_id=1) as ctx:
# Perform operations within the context
datasets = list(ctx.datasets)
for dataset in datasets:
print(dataset.name)
Managing Datasets
Listing Datasets
with dx.installation(installation, workspace_key=workspace_key) as ctx:
for dataset in ctx.datasets:
print(dataset.name, dataset.id)
Accessing Dataset Operations
Dataset operations may be accessed by name if the dataset name is unique within the installation, or by dataset_id.
with dx.installation(installation, workspace_key=workspace_key) as ctx:
dataset_ops = ctx.datasets.find(name='My Dataset')
with dx.installation(installation, workspace_key=workspace_key) as ctx:
dataset_ops = ctx.datasets.find(dataset_id='123adatasetid')
with dx.installation(installation, workspace_key=workspace_key) as ctx:
dataset_ops = ctx.datasets.find(dataset_id=UUID('123adatasetid'))
Creating a Dataset
from mig_dx_api import DatasetSchema, SchemaProperty
# Define the schema
schema = DatasetSchema(
properties=[
SchemaProperty(name='my_string', type='string', required=True),
SchemaProperty(name='my_integer', type='integer', required=True),
SchemaProperty(name='my_boolean', type='boolean', required=False),
],
primary_key=['my_string']
)
# Create the dataset
with dx.installation(installation, workspace_key=workspace_key) as ctx:
new_dataset = ctx.datasets.create(
name='My Dataset',
description='A test dataset',
schema=schema.model_dump() # this can also be defined as a dictionary
)
Creating a Dataset with specific tags
# Get tag options
with dx.installation(installation, workspace_key=workspace_key) as ctx:
tag_options = ctx.datasets.get_tags()
tags = [tags['data'][0]['movementAppId'], tags['data'][1]['movementAppId']]
new_dataset = ctx.datasets.create(
name='My Tagged Dataset',
description='A test tagged dataset',
schema={
"primaryKey": ["my_van_id"],
"properties": [
{"name": "my_van_id", "type": "string"},
{"name": "first_name", "type": "string"},
{"name": "last_name", "type": "string"},
],
},
tag_ids=[tags]
)
Uploading Data to a Dataset
data = [
{'my_string': 'string1', 'my_integer': 1, 'my_boolean': True},
{'my_string': 'string2', 'my_integer': 2, 'my_boolean': False},
{'my_string': 'string3', 'my_integer': 3, 'my_boolean': True},
]
with dx.installation(installation, workspace_key=workspace_key) as ctx:
dataset_ops = ctx.datasets.find(name='My Dataset')
dataset_ops.load(data, validate_records=True) # validate_records=True will validate the records against the schema using Pydantic
Load defaults to csv for newline-delimited json (NDJSON), but can support optional content types of tsv or other json types if the type is passed in, e.g. dataset_ops.load(data, validate_records=True, content_type='json')
While the load default upload_mode is create, append and replace are also available to add to a dataset or replace a dataset entirely, respectively.
Dataset Jobs
with dx.installation(installation, workspace_key=workspace_key) as ctx:
dataset_ops = ctx.datasets.find(name='My Dataset')
# Job Id comes from load, load_from_file or load_from_url
job_id = dataset_ops.load(data, validate_records=True)["jobId"]
# Get dataset job
job = ctx.datasets.get_dataset_job(job_id)
# Get logs for dataset job
job_logs = ctx.datasets.get_dataset_job_logs(job_id)
# Get latest status for dataset job
job_logs_current = ctx.datasets.get_current_dataset_logs(job_id)
Retrieving Records from a Dataset
with dx.installation(installation, workspace_key=workspace_key) as ctx:
dataset_ops = ctx.datasets.find(name='My Dataset')
records = dataset_ops.records()
for record in records:
print(record)
Deleting A Dataset
with dx.installation(installation, workspace_key=workspace_key) as ctx:
dataset_ops = ctx.datasets.delete(dataset_id=my_dataset_id)
Asynchronous Usage
The library supports asynchronous operations using async/await.
import asyncio
async def main():
dx = DX()
async with dx.installation(installation, workspace_key=workspace_key) as ctx:
async for dataset in ctx.datasets:
print(dataset.name)
dataset = await ctx.datasets.find(name='My Dataset')
data = [
{'my_string': 'string1', 'my_integer': 1, 'my_boolean': True},
{'my_string': 'string2', 'my_integer': 2, 'my_boolean': False},
{'my_string': 'string3', 'my_integer': 3, 'my_boolean': True},
]
await dataset.load(data)
async for record in dataset.records():
print(record)
asyncio.run(main())
Examples
Example: Loading Data from a File
with dx.installation(installation, workspace_key=workspace_key) as ctx:
dataset_ops = ctx.datasets.find(name='My Dataset')
dataset_ops.load_from_file('data.csv')
TSV and JSON files are also supported. Uses file extension to determine file type.
Example: Uploading Data from a URL
with dx.installation(installation, workspace_key=workspace_key) as ctx:
dataset_ops = ctx.datasets.find(name='My Dataset')
dataset_ops.load_from_url('https://example.com/data.csv', mode="create")
TSV and JSON files are also supported. Uses file extension to determine file type.
License
MIT License. See License
Note: This README assumes that the package name is mig-dx-api and that the code is properly packaged and available for installation via pip. Adjust the instructions accordingly based on the actual package name and installation method.
Metadata
Release files for mig-dx-api 0.2.12
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| mig_dx_api-0.2.12.tar.gz | 65.1 kB | Without provenance | Details for mig_dx_api-0.2.12.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| mig_dx_api-0.2.12-py3-none-any.whl | Python 3 | none | any | Details for mig_dx_api-0.2.12-py3-none-any.whl |
Total release size: 80.9 kB
Release files / mig_dx_api-0.2.12.tar.gz
| Download URL | mig_dx_api-0.2.12.tar.gz |
|---|---|
| Size | 65.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
16613b0e127663727aa4f0ff56591e7d7352bf15559c3ccc54a1de20adb954af
|
|
BLAKE2b-256 checksum How to use checksums |
786f1c54c8e1b1580f6cf4d0f93da12b41c844c0b8d79c15e4a9c5d7550b3f39
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.7
|
Release files / mig_dx_api-0.2.12-py3-none-any.whl
| Download URL | mig_dx_api-0.2.12-py3-none-any.whl |
|---|---|
| Size | 15.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
28ac7ab81d818a4b10c4d21c58131bb2abd35ea317a3c736708b4b2f69dfeafe
|
|
BLAKE2b-256 checksum How to use checksums |
eda904b94ec2d1d59ee4b720f6969d8b666b9ac9f55f39fcd67def901bd79e40
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.7
|
Security for mig-dx-api 0.2.12
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.