etiket-sync-agent-qualibrate 0.0.1

QUAlibrate connector for eTiKeT sync agent

eTiKeT Sync Agent - QUAlibrate Connector

Connector for synchronizing QUAlibrate calibration data with the eTiKeT platform. This connector scans a QUAlibrate data directory for node/workflow snapshots and syncs them to the cloud.

Installation

Install the QUAlibrate connector using the eTiKeT Sync SDK:

from etiket_sdk.sync import Connectors

# Install the latest version
Connectors.install_from_pypi("etiket-sync-agent-qualibrate")

The package is automatically discovered by etiket_sync_agent through the entry-point system. Once installed, you can verify with:

# List installed connectors
print(Connectors.list())

# Get details for the QUAlibrate connector
connector = Connectors.get("etiket_sync_agent_qualibrate")
print(connector)

Updating the Connector

# Update to the latest version
Connectors.update_from_pypi("etiket-sync-agent-qualibrate")

What Gets Synchronized

Each QUAlibrate snapshot is a directory containing a node.json (metadata) and a data.json (results, with references to .npz/.h5/.png/.json files). When a snapshot is synced, the following data is extracted and uploaded:

QUAlibrate Data	eTiKeT Field	Description
Snapshot folder name	alt_uid	Unique identifier for the snapshot
metadata.name (or folder name)	name	Name of the dataset
created_at / metadata.run_start	collected	When the snapshot was created
metadata.type_of_execution	keywords	node or workflow
metadata.description	description	Snapshot description
type_of_execution, status	attributes	Small, searchable scalar metadata
data.json → raw_data	HDF5 file	Raw measurement arrays combined into a single netCDF (raw_data.h5)

Data Processing

• Reference resolution: data.json reference strings (./arrays.npz#a.b.c) are resolved into real numpy arrays, xarray datasets, and images via the vendored loader.
• Raw-data extraction: Only the raw_data entry of the results tree is converted to netCDF. Other entries (e.g. fit_results) are intentionally excluded.
• Array-to-dataset convention (npz format): this is a best-effort reconstruction. Within a group of sibling numpy arrays, the leading 1-D arrays are treated as the coordinate axes in order (x, y, z, ...) and the remaining arrays as the measured values over the grid those axes span. If shapes don't fit this convention, arrays fall back to standalone variables with independent dimensions.
• Combining: per-measurement datasets are merged into one flat netCDF. Data variables are prefixed by their results-path; shared sweep axes stay shared (join="exact"). On conflict, every dataset is fully namespaced (lossless, collision-proof).
• Workflows: workflow snapshots typically carry no array data, in which case no netCDF file is uploaded.

---

Configuration

The QUAlibrate connector requires a QualibrateConfigData configuration with the following fields:

Field	Type	Required	Description
data_directory	Path or str	Yes	Path to the QUAlibrate data directory (the root containing snapshot folders)
is_server_folder	bool	Yes	Whether this is a server folder (e.g. on a network drive of the university)

A scope is required for this connector (scope_requirement = REQUIRED); scope mapping is not supported.

The configuration is validated on creation: data_directory must exist and be a directory, and it must not overlap (be identical to, a parent of, or a subdirectory of) the path of an existing QUAlibrate sync source.

Example Configuration

Example using the etiket-sdk package:

from etiket_sdk.sync import SyncSources

SyncSources.create(
    name="my_qualibrate_source",
    connector_identifier="etiket_sync_agent_qualibrate",
    config_data={
        "data_directory": "~/.qualibrate/user_storage",
        "is_server_folder": False
    },
    default_scope="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
)

Live Sync

Live synchronization is not supported by this connector. Snapshots are synced once they are complete (i.e. once a node.json is present in the snapshot directory).

Requirements

• Python >= 3.10
• numpy
• xarray
• h5netcdf
• Pillow

License

Copyright © 2025 QHarbor. All Rights Reserved. See LICENCE for details.