deep-learning-azure 0.0.12

Generic Azure integration layer for dl-core.

deep-learning-azure

Public Azure integration layer for deep-learning-core.

deep-learning-azure adds Azure ML execution, Azure storage helpers, and Azure-oriented dataset wrappers on top of deep-learning-core.

Install it directly or through the deep-learning-core[azure] extra. The package is kept separate so Azure-specific dependencies and scaffold wiring do not leak into plain deep-learning-core installations.

Install

Install from PyPI through the core extra:

pip install "deep-learning-core[azure]"

Install the package directly:

pip install deep-learning-azure

Install in a uv project:

uv add "deep-learning-core[azure]" deep-learning-azure

Scope

• Azure ML executor
• Azure storage helpers and AzCopy wrappers
• Azure dataset wrappers
• Azure experiment scaffold integration through dl-init --with-azure

Out Of Scope

• Generic trainer, dataset, and metric abstractions
• Public framework defaults
• Concrete experiment repositories

Quick Start

Install it into an experiment repository through the Azure extra:

uv add "deep-learning-core[azure]" deep-learning-azure

If the repository was scaffolded with dl-init --with-azure, the experiment package will import dl_azure automatically so its executor and generic dataset wrappers register at runtime, and the scaffold will also create azure-config.json.

The Azure executor is sweep-oriented. Use uv run dl-sweep experiments/lr_sweep.yaml --dry-run before the first real submission in a new repository.

Concrete experiment flow:

uv init
uv add deep-learning-azure
uv run dl-init --root-dir . --with-azure
uv run dl-core add dataset AzureSeq --base azure_compute_multiframe
uv run dl-sweep experiments/lr_sweep.yaml --dry-run

Tracker naming defaults to the repository root name. If you want Azure job submission and Azure MLflow to use a different destination name, set tracking.experiment_name in your sweep config.

Azure submissions automatically rewrite the default local runtime.output_dir from artifacts to outputs/artifacts inside the remote job. That keeps checkpoints, plots, metrics, and other run files under Azure ML's managed output directory without changing the local default artifact layout.

When you analyze an Azure-backed sweep with dl-analyze, the Azure metrics source fetches only the metric histories requested on the CLI, for example:

uv run dl-analyze --sweep experiments/lr_sweep.yaml \
  --metric test/eer --mode min \
  --metric test/accuracy --mode max \
  --rank-method rank-sum

Those fetched metric histories are cached in analysis_cache.json next to sweep_tracking.json. Use --force to refresh them.

If you want the tracked Azure job outputs locally after the sweep finishes, run:

uv run dl-sync --sweep experiments/lr_sweep.yaml --artifacts

That downloads the Azure job bundle for each tracked run and patches sweep_tracking.json with the resolved local artifact paths.

Concrete dataset scaffold examples:

uv run dl-core add dataset AzureImages --base azure_compute
uv run dl-core add dataset AzureFrames --base azure_compute_frame
uv run dl-core add dataset AzureSeq --base azure_compute_multiframe
uv run dl-core add dataset AzureStream --base azure_streaming
uv run dl-core add dataset AzureStreamSeq --base azure_streaming_multiframe

What You Get

• the azure executor
• Azure storage helpers and AzCopy wrappers
• generic Azure dataset foundations: AzureComputeWrapper, AzureStreamingWrapper, AzureComputeFrameWrapper, AzureStreamingFrameWrapper, AzureComputeMultiFrameWrapper, and AzureStreamingMultiFrameWrapper
• dl-init --with-azure scaffold integration
• a managed .amlignore block that preserves user content while excluding common local-only outputs from Azure submissions
• Azure job output routing to outputs/artifacts for automatic artifact persistence in Azure ML

Companion Packages

• dl-core
• dl-mlflow
• dl-wandb

Documentation

• Documentation Index
• GitHub Repository

License

MIT. See LICENSE.