azure-pipeline-validator 0.1.5

Validation-first Azure DevOps YAML validator with Azure preview and Azure LSP checks

Local-first Azure DevOps YAML validation with Azure-authoritative checks and Azure LSP parity by default.

Lifecycle
Core Stack
Navigation

1. Install uv if needed: curl -LsSf https://astral.sh/uv/install.sh | sh
2. Clone and enter the repo: git clone https://github.com/andrewmaspero/azure-pipeline-validator.git && cd azure-pipeline-validator
3. Install dependencies: uv sync --dev
4. Run the CLI: uv run azure-pipeline-validator validate --help

Package usage without cloning:

uvx azure-pipeline-validator validate --help

Feature Badge	Details
	Calls Azure DevOps pipeline preview API and returns real validationResults and finalYaml.
	Runs the Azure DevOps pipeline language server (LSP) for diagnostics aligned with editor behavior.
	Default gate mode is authoritative; optional strict mode is all.
	Optional style and structure checks, disabled by default.
	Generic schema checks remain available via --run-schema with deprecation warning.
	Rich text output plus machine-readable formats for CI consumers.
	Detects template shape and wraps steps/jobs/stages when needed.

Name	Required	Default	Format	Description
AZDO_ORG	yes (preview)	-	URL	Azure DevOps org URL, for example https://dev.azure.com/contoso.
AZDO_PROJECT	yes (preview)	-	string	Project name that owns the pipeline definition.
AZDO_PIPELINE_ID	yes (preview)	-	integer	Existing pipeline ID used for preview expansion.
AZDO_PAT	yes (preview, unless CI token)	-	string	PAT with Build Read and Execute permissions, or SYSTEM_ACCESSTOKEN in CI.
AZDO_REFNAME	no	refs/heads/main	string	Git ref used while expanding templates during preview.
AZDO_TIMEOUT_SECONDS	no	30	integer	Timeout for Azure preview API calls.

Name	Required	Default	Format	Description
AZP_VALIDATOR_LSP_OFFLINE	no	false	boolean	Forces cache-only Azure LSP asset usage.
AZP_VALIDATOR_LSP_CACHE_DIR	no	~/.azure-pipeline-validator/lsp-assets	path	Cache directory for language-server assets.
AZP_VALIDATOR_LSP_VERSION	no	latest	string	Pins downloaded Azure Pipelines language-server asset version.
AZP_VALIDATOR_LSP_SHA256	no	-	hex	Optional checksum verification for language-server asset download.
AZP_VALIDATOR_LSP_DOWNLOAD_TIMEOUT_SECONDS	no	30	integer	Timeout for language-server asset download operations.
AZP_VALIDATOR_NODE_VERSION	no	lts	string	Node.js version for auto-install (lts, latest, or exact semver).
AZP_VALIDATOR_NODE_CACHE_DIR	no	~/.azure-pipeline-validator/node-runtime	path	Cache directory for downloaded Node.js runtimes.
AZP_VALIDATOR_NODE_DOWNLOAD_TIMEOUT_SECONDS	no	30	integer	Timeout for Node.js download operations.

Notes:

• Default behavior runs preview and lsp; these require Azure credentials and a Node runtime.
• If node is not present on PATH, the CLI auto-downloads a compatible Node runtime into the user cache.
• yamllint and schema are advisory by default and disabled until explicitly enabled.
• If both authoritative stages are disabled while --gate-mode authoritative is requested, gating falls back to all with a warning.

Usage: azure-pipeline-validator validate [OPTIONS] [PATH]

Run authoritative Azure validation by default (preview + lsp), with optional advisory yamllint/schema stages.

Arguments:
  PATH  File or directory to validate. Directories are scanned recursively for *.yml and *.yaml files.  [default: .]

Options:
  --repo-root PATH                     Base path used when resolving template references (defaults to CWD).
  --run-yamllint / --skip-yamllint     Enable or disable optional advisory yamllint checks.  [default: skip-yamllint]
  --run-schema / --skip-schema         Enable or disable deprecated advisory schema checks.  [default: skip-schema]
  --run-preview / --skip-preview       Call the Azure DevOps preview endpoint to fetch the compiled finalYaml.
  --run-lsp / --skip-lsp             Validate via Azure DevOps pipeline language server (LSP).
  --lsp-server-path PATH             Path to language server entrypoint (dist/server.js).
  --lsp-schema-path PATH             Path to language server schema (service-schema.json).
  --lsp-timeout-seconds SECONDS     Diagnostics wait timeout per file.  [default: 5.0]
  --output-format FORMAT               Reporter output format.  [default: text]
  --gate-mode MODE                     Blocking policy for exit code: authoritative|all.  [default: authoritative]
  --hidden-mode MODE                   Hidden directory discovery: common|all|none.  [default: common]
  --fail-fast / --no-fail-fast         Stop immediately after the first file that fails validation.
  --help                               Show this message and exit.

Hidden directories

Default hidden discovery mode is common, which auto-discovers Azure DevOps-oriented hidden directories such as .azure-pipelines, .azure-pipeline, .azure-pipelines-templates, .azure-pipeline-templates, .azure_pipelines, .azure_pipeline, .azure_pipeline_templates, .azure, .azure-devops-pipelines, .azuredevops-pipelines, .azure_devops, .azure_devops_pipelines, .devops, .devops-pipelines, .devops-pipeline, .devops-templates, .devops-ci, .devops-cicd, .ado, .ado-pipelines, .ado-pipeline, .ado-templates, .azdo, .azdo-pipelines, .azdo-pipeline, .azdo-templates, .azuredevops, .azure-devops, .azpipelines, .azp, .azp-pipelines, .azp-templates, .pipelines, .pipeline, .pipelines-templates, .pipeline-templates, .release-pipelines, .build-pipelines, .ci, .cicd, .ci-cd.

Examples:

# default (common): includes common Azure DevOps hidden directories
uv run azure-pipeline-validator validate . --repo-root $(pwd)

# strict: skip hidden directories during directory scans
uv run azure-pipeline-validator validate --hidden-mode none . --repo-root $(pwd)

# broad: include all hidden directories (except hard exclusions like .git/.github)
uv run azure-pipeline-validator validate --hidden-mode all . --repo-root $(pwd)

# explicit hidden target is included in common mode
uv run azure-pipeline-validator validate .devops/ --repo-root $(pwd)

• text (default): rounded table with per-stage status (pass, skip, error, or first finding).
• json: one stable payload containing schema_version, summary, and files.
• ndjson: one record per file plus one summary record.

Example text output:

╭──────────────────────┬──────────┬────────┬──────────────────────┬──────────────────────╮
│ File                 │ yamllint │ schema │ preview              │ lsp                  │
├──────────────────────┼──────────┼────────┼──────────────────────┼──────────────────────┤
│ workflows/ci.yml     │ pass     │ skip   │ pass                 │ pass                 │
│ workflows/deploy.yml │ skip     │ skip   │ pass                 │ L12 C9: pattern ...  │
╰──────────────────────┴──────────┴────────┴──────────────────────┴──────────────────────╯
Validated 2 file(s). Blocking failures: 1. Advisory-only files: 0. Gate mode: authoritative.

This repo uses three GitHub workflows:

1. CI
   • PR semantic-title enforcement.
   • Ruff, format check, pytest, and coverage gate (>=90%).
2. Release Please
   • Conventional commits drive semantic versioning.
   • Updates pyproject.toml version, creates tag, and creates GitHub Release.
3. Publish
   • Publishes to PyPI via Trusted Publishing.
   • Publishes OCI dist artifact to GHCR for Packages visibility.

Setting	Value
PyPI project name	azure-pipeline-validator
GitHub owner	andrewmaspero
GitHub repository	azure-pipeline-validator
Workflow file	publish.yml
Environment name	pypi

# Install dependencies
uv sync --dev

# Lint and format checks
uv run ruff check
uv run ruff format --check

# Test suite
uv run python -m pytest

# Coverage gate
uv run python -m pytest --cov=src/azure_pipelines_validator --cov-report=term-missing --cov-fail-under=90

# Google docstring style checks
uvx --from pydocstyle pydocstyle --convention=google src

• Validate a repo quickly: uv run azure-pipeline-validator validate . --repo-root $(pwd)
• Authoritative-only gate (default): preview and lsp determine exit code.
• Strict all-stage gate: --run-yamllint --run-schema --gate-mode all
• Fail fast for CI cost control: add --fail-fast.
• Treat schema warnings as advisory; schema stage is soft-deprecated for Azure correctness.

• CLI and orchestration: Typer-based command surface and validation service pipeline.
• Azure-authoritative engines:
  • Azure DevOps Preview API (yamlOverride against existing pipeline definition).
  • Azure DevOps pipeline language server (LSP) diagnostics.
• Advisory engines:
  • yamllint for style and structure.
  • JSON schema checks for generic coverage.
• Output and gating model:
  • Stage-level statuses: passed, failed, error, skipped.
  • Gate modes: authoritative and all.
  • Reporter supports text, JSON, and NDJSON for CI ingestion.

• Symptom: many Red Hat YAML warnings but one Azure error.
  • Cause: generic YAML schema validation and Azure template semantics diverge.
  • Action: rely on preview and lsp as source of truth for Azure correctness.
• Symptom: Azure LSP stage cannot start.
  • Cause: unavailable Node runtime or language-server assets.
  • Action: rerun with network access so runtime/assets can auto-bootstrap, or provide --lsp-server-path and --lsp-schema-path plus a valid Node binary.
• Symptom: preview failures in CI.
  • Cause: missing Azure credentials or insufficient PAT scope.
  • Action: ensure AZDO_* values are present and token has Build Read and Execute permissions.

MIT. See LICENSE.