current-data-py

Python SDK for robotics Protobuf messages, TD files, and platform service clients. Run shell commands from the repository root in the shared development environment.

Use locally

Run the bundled examples to write and read a local TD file using the installed SDK:

python -I -m current_data_py.examples.write_td /tmp/current-data-example.td
python -I -m current_data_py.examples.read_td /tmp/current-data-example.td

-I avoids importing the unbuilt source directory in place of the installed package. See examples for other entry points.

For a minimal read-only service call, run this snippet with python -I so it uses the installed SDK:

from contextlib import closing
import os
os.environ["ENV"] = "test"

from current_data_py.management import ManagementClient, MetadataFilter

with ManagementClient() as client:
    with closing(client.stream_data_references(MetadataFilter(), timeout=30)) as records:
        record = next(records, None)
        print(record.id if record is not None else "No matching data")

Set ENV=test before the first SDK import to select the test service; an unset or empty value defaults to production. An explicit client address= overrides the default. Remote examples require service connectivity and accessible data.

Upload files

Use OSSClient or TOSClient with upload permission. The management service selects the bucket and allowed object prefix; keep it open during transfers:

from current_data_py.management import ManagementClient
from current_data_py.tos import TOSClient

with ManagementClient() as management:
    with TOSClient(management, permission="upload", data_type="YAM",
                   data_kind="CONVERTED") as storage:
        storage.upload_file(storage.object_prefix + "recording.td", "/data/recording.td",
                            multipart_threshold=100 * 1024**2,
                            part_size=16 * 1024**2, max_workers=4)

Both clients accept the same transfer options. Files at or above multipart_threshold use multipart upload; 0 forces multipart for nonempty files. Empty files always use PutObject. Sizes are in bytes. part_size must be 5 MiB–5 GiB, and the file must fit within 10,000 parts; increase it for larger files. max_workers controls parallel requests (1–64); part buffers use roughly part_size * max_workers memory, plus SDK overhead. Do not modify the source file during upload.

Uploads reject existing objects by default, including when completing multipart uploads. Set forbid_overwrite=False to replace one. Content type and metadata are preserved. The return value is the provider's PutObject result for small files or CompleteMultipartUpload result for multipart files.

Failed multipart transfers attempt to abort and delete incomplete parts. A failed cleanup emits a warning; check the bucket's incomplete uploads. The upload role and its scoped STS policy must allow oss:AbortMultipartUpload or tos:AbortMultipartUpload in addition to PutObject. Deploy the management-service policy update and ensure the underlying cloud role permits cleanup before use. There is no persistent checkpoint: a later call starts a new upload. Uploading does not register the data; call ManagementClient.register_data separately.

Test changes

The package bundles generated Protobuf modules during its build. After protocol changes, regenerate the bindings first. Build into a fresh temporary directory, validate the artifacts, and install this checkout's wheel before running its checks. Only the local package is installed; shared dependencies are left unchanged:

(
  set -e
  sdk_dist=$(mktemp -d)
  trap 'rm -rf "$sdk_dist"' EXIT
  (cd current_data_py && python -m build --no-isolation --outdir "$sdk_dist")
  python current_data_py/tests/check_distribution.py --dist-dir "$sdk_dist"
  python -m pip install --no-deps --force-reinstall "$sdk_dist"/*.whl
  for check in package environment management labeling remote oss tos decoding multipart docs; do
    python -I "current_data_py/tests/check_${check}.py"
  done
)

These checks cover packaging, environment selection, clients, and media decoding using local fixtures and mocked services. They do not require production credentials or prove connectivity to a deployed service.

Publish to PyPI

CI checks this SDK but does not publish it. Releases are manual and must use the latest successfully fetched origin/main, including the version bump. Development branches, uncommitted changes, and older main commits are not release sources. Follow the repository Git workflow. Prepare the version update in pyproject.toml on a development branch and merge it through review together with the intended SDK changes before building a release. Use a version that has not already been published.

Build and validate

Run from the repository root. Fetch the latest origin/main, record its commit, and build from a clean exported copy so local edits and old artifacts cannot enter the release:

sdk_release_dir=$(mktemp -d /tmp/current-data-release.XXXXXX)
(
  set -euo pipefail
  git fetch origin
  sdk_release_sha=$(git rev-parse origin/main)
  printf '%s\n' "$sdk_release_sha" > "$sdk_release_dir/source-commit.txt"
  mkdir "$sdk_release_dir/source"
  git archive "$sdk_release_sha" | tar -x -C "$sdk_release_dir/source"
  (cd "$sdk_release_dir/source/current_data_py" && python -m build --no-isolation --outdir "$sdk_release_dir/dist")
  python "$sdk_release_dir/source/current_data_py/tests/check_distribution.py" --dist-dir "$sdk_release_dir/dist"
  python -m twine check "$sdk_release_dir/dist/"*
  python -m pip install --no-deps --force-reinstall "$sdk_release_dir/dist/"*.whl
  (
    cd "$sdk_release_dir"
    unset PYTHONPATH PYTHONHOME
    for check in package environment management labeling remote oss tos decoding; do
      python -I "$sdk_release_dir/source/current_data_py/tests/check_${check}.py"
    done
  )
  (cd "$sdk_release_dir/dist" && sha256sum *) > "$sdk_release_dir/SHA256SUMS"
)

Release checks run outside both source trees so child Python processes also use the installed wheel. They come from the recorded origin/main commit; unmerged local test fixes are not included in the export.

Continue only after every check succeeds. Confirm source-commit.txt contains the intended main commit and inspect the wheel for the intended release features. Review the built package's README metadata for public display; exclude private deployment addresses, bucket names, and credentials. Keep the release directory and use these exact validated artifacts for uploading.

Upload

Use your configured PyPI credentials or Twine's interactive token prompt; do not put tokens in the README or shell commands. Run from the repository root in the same shell that defines sdk_release_dir.

Fetch main again immediately before uploading. If its commit has changed, rebuild and rerun all checks from the new latest main. The command also checks that the artifacts still match the hashes recorded after validation:

(
  set -e
  sdk_repository=pypi
  git fetch origin
  if [ "$(git rev-parse origin/main)" != "$(cat "$sdk_release_dir/source-commit.txt")" ]; then
    echo "origin/main changed; rebuild and validate the latest main before uploading." >&2
    exit 1
  fi
  (cd "$sdk_release_dir/dist" && sha256sum -c ../SHA256SUMS)
  python -m twine upload --repository "$sdk_repository" "$sdk_release_dir/dist/"*
)

For an optional TestPyPI trial, change sdk_repository=pypi to sdk_repository=testpypi in the same block and use TestPyPI credentials. Both registries require the latest-main and artifact checks.

Verify the release

After upload, verify that PyPI shows the expected version and rendered description. Compare the SHA-256 digests of both published files with SHA256SUMS, and retain that file with source-commit.txt as the release record. An optional TestPyPI trial does not replace verification of the production PyPI release.