kedro-azureml-pipeline 0.1.0a4

What is Kedro AzureML Pipeline?

Kedro AzureML Pipeline is a plugin that enables running Kedro pipelines on Azure ML Pipelines. It translates your Kedro pipeline into an Azure ML pipeline job where each Kedro node becomes a separate step.

Two deployment workflows are supported, both backed by Azure ML Environments:

• Code upload: only dependencies live in the Docker image; source code is uploaded at runtime (fast iteration for data scientists)
• Docker image: code is baked into the image (stable, repeatable workflows for MLOps)

Key features

Installation

pip install kedro-azureml-pipeline

or with uv:

uv add kedro-azureml-pipeline

Quick start

1. Initialize configuration

kedro azureml init

This creates conf/base/azureml.yml with placeholder values and an .amlignore file.

2. Review the generated configuration

Open conf/base/azureml.yml and fill in your Azure details:

workspace:
  __default__:
    subscription_id: "<subscription_id>"
    resource_group: "<resource_group>"
    name: "<workspace_name>"

compute:
  __default__:
    cluster_name: "<cluster_name>"

execution:
  environment: "<environment>"
  code_directory: "."

3. Define a job and submit

Add a job to azureml.yml:

jobs:
  training:
    pipeline:
      pipeline_name: "__default__"
    experiment_name: "my-experiment"

Then submit it:

kedro azureml submit -j training

Use --dry-run to preview without submitting, or --wait-for-completion to block until the run finishes.

4. Compile to YAML (optional)

Export the Azure ML pipeline definition for inspection or CI:

kedro azureml compile -j training -o pipeline.yaml

Documentation

Full documentation is available at https://kedro-azureml-pipeline.readthedocs.io/.

Contributing

We welcome contributions, feedback, and questions:

• Report issues or request features: GitHub Issues
• Contributing guide: CONTRIBUTING.md
• Discussions: GitHub Discussions

License

This project is licensed under the terms of the Apache-2.0 License.

How do I cite Kedro AzureML Pipeline?

If you use Kedro AzureML Pipeline in work you publish, please cite it:

Guillaume Tauzin. Kedro AzureML Pipeline: Orchestrate Kedro pipelines with Azure ML Pipeline. https://github.com/stateful-y/kedro-azureml-pipeline

Or in BibTeX:

@software{kedro_azureml_pipeline,
  author  = "Guillaume Tauzin",
  title   = "{Kedro AzureML Pipeline: Orchestrate Kedro pipelines with Azure ML Pipeline}",
  url     = "https://github.com/stateful-y/kedro-azureml-pipeline",
  license = "Apache-2.0"
}

Reference managers can read CITATION.cff directly. To cite a specific version, see the citation page.

Acknowledgements

This project is a fork of kedro-azureml, originally developed by GetInData. We are grateful for their work in creating the initial plugin that bridges Kedro and Azure ML Pipelines. We have continued development to add new features, improve documentation, and maintain the project under the kedro-azureml-pipeline package name.

We would also like to thank Evolta Technologies for their support to the project.

This project is maintained by stateful-y, an ML consultancy specializing in MLOps and data science & engineering. If you're interested in collaborating or learning more about our services, please visit our website.