eTiKeT Sync Agent - QUAlibrate Connector

Connector for synchronizing QUAlibrate calibration data with the eTiKeT platform. This connector scans a QUAlibrate data directory for node/workflow snapshots and syncs them to the cloud.

Installation

Install the QUAlibrate connector using the eTiKeT Sync SDK:

from etiket_sdk.sync import Connectors

# Install the latest version
Connectors.install_from_pypi("etiket-sync-agent-qualibrate")

The package is automatically discovered by etiket_sync_agent through the entry-point system. Once installed, you can verify with:

# List installed connectors
print(Connectors.list())

# Get details for the QUAlibrate connector
connector = Connectors.get("etiket_sync_agent_qualibrate")
print(connector)

Updating the Connector

# Update to the latest version
Connectors.update_from_pypi("etiket-sync-agent-qualibrate")

What Gets Synchronized

Each QUAlibrate snapshot is a directory containing a node.json (metadata) and a data.json (results, with references to .npz/.h5/.png/.json files). When a snapshot is synced, the following data is extracted and uploaded:

Data Processing

• Reference resolution: data.json reference strings (./arrays.npz#a.b.c) are resolved into real numpy arrays, xarray datasets, and images via the vendored loader.
• Raw-data extraction: Only the raw_data entry of the results tree is converted to netCDF. Other entries (e.g. fit_results) are intentionally excluded.
• Array-to-dataset convention (npz format): this is a best-effort reconstruction. Within a group of sibling numpy arrays, the leading 1-D arrays are treated as the coordinate axes in order (x, y, z, ...) and the remaining arrays as the measured values over the grid those axes span. If shapes don't fit this convention, arrays fall back to standalone variables with independent dimensions.
• Combining: per-measurement datasets are merged into one flat netCDF. Data variables are prefixed by their results-path; shared sweep axes stay shared (join="exact"). On conflict, every dataset is fully namespaced (lossless, collision-proof).
• Workflows: workflow snapshots typically carry no array data, in which case no netCDF file is uploaded.

Configuration

The QUAlibrate connector requires a QualibrateConfigData configuration with the following fields:

A scope is required for this connector (scope_requirement = REQUIRED); scope mapping is not supported.

The configuration is validated on creation: data_directory must exist and be a directory, and it must not overlap (be identical to, a parent of, or a subdirectory of) the path of an existing QUAlibrate sync source.

Example Configuration

Example using the etiket-sdk package:

from etiket_sdk.sync import SyncSources

SyncSources.create(
    name="my_qualibrate_source",
    connector_identifier="etiket_sync_agent_qualibrate",
    config_data={
        "data_directory": "~/.qualibrate/user_storage",
        "is_server_folder": False
    },
    default_scope="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
)

Live Sync

Live synchronization is not supported by this connector. Snapshots are synced once they are complete (i.e. once a node.json is present in the snapshot directory).

Requirements

• Python >= 3.10
• numpy
• xarray
• h5netcdf
• Pillow

License

Copyright © 2025 QHarbor. All Rights Reserved. See LICENCE for details.