pycli

A lightweight Python-compatible DevOps DSL that extends Python with first-class shell command execution.

Overview

pycli transpiles .spy files into pure standard Python (.py), providing:

• Python Readability: Natural Python syntax and standard ecosystem compatibility.
• PowerShell-like Command Invocation: Run shell commands directly with $(...).
• Bash-like Command Composition: Pipelines (|), redirections (>, >>, <), and subcommands.
• Native Object Handling: Seamless integration with Python objects, string interpolation {var}, and list expansion ({*files}).
• Cross-Platform Compatibility: Tested and verified across Linux (ubuntu-latest), macOS (macos-latest), and Windows (windows-latest).
• Zero Custom VM: Transpiled directly to standard Python and executed on standard CPython.

See the full specification in docs/pycli-grammar.md.

Example

Source DSL (deploy.spy)

target_env = "production"
branch = $(git branch --show-current).text

# 1. Pipelines (|) and line streaming
live_pods = $(kubectl get pods -n {target_env} | grep -E 'Running|Pending').lines
for pod in live_pods:
    print(f"Active pod: {pod}")

# 2. Context managers: temporary directory (cd) and environment variables (env)
with cd("frontend"), env(NODE_ENV=target_env):
    $(npm ci)!
    build_log = $(npm run build).tee

# 3. List expansion (splat), structured JSON output, and safe probing (?)
artifacts = ["dist/app.js", "dist/app.css"]
$(gzip -k {*artifacts})

cluster = $(az aks show --name prod-cluster --resource-group {target_env}).json
print(f"Cluster FQDN: {cluster.fqdn}")

status = $(curl -sSf http://localhost:8080/health)?
if status:
    print("Health check passed successfully!")

Transpiled Python (deploy.py)

from pycli.runtime import cd, env, run, run_expanded

target_env = "production"
branch = run("git branch --show-current").text

# 1. Pipelines (|) and line streaming
live_pods = run(f"""kubectl get pods -n {target_env} | grep -E 'Running|Pending'""").lines
for pod in live_pods:
    print(f"Active pod: {pod}")

# 2. Context managers: temporary directory (cd) and environment variables (env)
with cd("frontend"), env(NODE_ENV=target_env):
    run("npm ci", capture=False, check=True)
    build_log = run("npm run build", tee=True)

# 3. List expansion (splat), structured JSON output, and safe probing (?)
artifacts = ["dist/app.js", "dist/app.css"]
run_expanded("gzip", "-k", *artifacts, capture=False)

cluster = run(f"az aks show --name prod-cluster --resource-group {target_env}").json
print(f"Cluster FQDN: {cluster.fqdn}")

status = run("curl -sSf http://localhost:8080/health", suppress_errors=True)
if status:
    print("Health check passed successfully!")

CLI Usage

Global Installation (Centralized Command)

You can install pycli globally into your system PATH using uv tool:

uv tool install --editable . --force

This registers two commands globally on your machine:

• spy: Ultra-concise runner for .spy scripts.
• pycli: The full CLI tool with subcommands.

Once installed, you can run .spy scripts from any folder or terminal:

spy script.spy
# or
pycli script.spy

Direct Script Execution without Global Install

If working inside this repository with uv:

uv run pycli script.spy

Transpile .spy to .py

# Output to stdout with syntax coloring
spy transpile script.spy

# Output to a file (clean Python without ANSI codes)
spy transpile script.spy -o script.py

# Force / disable color
spy transpile script.spy --color
spy transpile script.spy --no-color

# Validate generated Python code with ast.parse
spy transpile script.spy --validate

# Disable automatic shell_quote() sanitization on interpolations
spy transpile script.spy --unsafe-interpolation

Run .spy Scripts

# Run a script directly
spy run script.spy
# or simply
spy script.spy

# Run with generated Python syntax validation
spy run --validate script.spy

# Run with unsafe interpolation (disables shell_quote)
spy run --unsafe-interpolation script.spy

# Run with warning on untrusted external scripts
spy run --warn-external script.spy

---

Security & Robustness

Automatic Shell Interpolation Sanitization

By default, all variable interpolations {var} and dynamic redirection targets are wrapped with shell_quote(var) (shlex.quote) during transpilation. This protects against shell injection attacks if variables contain metacharacters (;, &&, |, etc.). If raw, unquoted shell syntax expansion is explicitly needed, pass --unsafe-interpolation or use transpile(..., unsafe_interpolation=True).

Execution Privilege Model (Not a Sandbox)

pycli executes .spy files on standard CPython runtimes with the full privileges and environment of the user running the process. It is not a sandbox. When executing .spy scripts from external or untrusted sources, use --warn-external and verify the script contents.

Command Execution Controls

The runtime functions run(), run_expanded(), and async_run() support robust controls:

• timeout: Terminate hanging processes and raise CommandTimeoutError.
• encoding: Custom text decoding (default "utf-8", configurable to CP1252, Latin-1, etc.).
• max_output_bytes: Cap memory consumption by truncating stdout/stderr beyond a threshold (res.truncated = True).

Supported Language Features

Feature	Syntax Example	Target Python Equivalent
Statement Form	$(git status)	run("git status", capture=False) (streams output to console)
Expression Form	res = $(git status)	res = run("git status") (captures stdout/stderr)
Strict Mode	$(git status)!	run("git status", capture=False, check=True)
Safe Mode	$(curl http://...) ?	run(..., suppress_errors=True) (no exception on non-zero exit code)
Background / Async	job = $(docker build .) &	run_bg(...) returning BackgroundJob (.wait(), .poll(), .kill())
Async / Await	res = await $(git pull)	await async_run(...) for asyncio workflows
Live Stream & Capture	res = $(npm test).tee	run(..., tee=True) (live streaming to console + captured in res)
Stdin Piping	$(kubectl apply -f -).input(yaml)	run(..., input=yaml)
Output Line Iteration	for line in $(git log): ...	Direct iteration over res, or res.lines and res.text
Context Managers	with cd(dir):, with env(K="V"):	Temporary directory and environment variable scoping
Quote Interpolation	$(echo "{name}" '{raw}')	Double quotes interpolate {expr}; single quotes stay strictly literal
List Expansion (Splat)	$(rm {*files})	run_expanded("rm", *files, capture=False)
Pipelines	$(kubectl get pods | grep api)	run("kubectl get pods | grep api", capture=False)
Redirection	$(git status > status.txt)	run("git status > status.txt", capture=False)
Subcommands	$(echo $(git branch --show-current))	run("echo $(git branch --show-current)", capture=False)
Truthiness	if $(git diff --quiet): ...	if run("git diff --quiet"): ... (truthy if exit_code == 0)
Structured Output (JSON)	vms = $(az vm list).json	Navigable DynamicObj via vm.name or vm["name"]
Interactive REPL	spy repl or spy	Interactive shell with on-the-fly transpilation
CommandResult Properties	res = $(git status)	res.stdout, res.stderr, res.exit_code, res.lines, res.text
Modular .spy Imports	import devops_utils	Seamlessly import .spy files and packages via Python importlib hook

---

Complete Language Syntax Reference

spy is a superset of standard Python. Everything that is valid in Python 3.12+ is fully valid in .spy. spy introduces the Command Expression $(...) for seamless command-line execution and shell orchestration.

1. Command Invocation Forms

Statement Form (Unassigned)

When a command expression appears as a standalone line or single-line statement:

$(terraform init)
if should_apply: $(terraform apply -auto-approve)

• Execution: The command is executed and its output (stdout and stderr) is streamed live to the console in real-time.
• Return value: Discarded (capture=False).

Expression Form (Assigned / Inline)

When a command expression is assigned to a variable, passed as a function argument, or used in an expression:

current_branch = $(git branch --show-current)
log_output = $(git log -n 10).text

• Execution: The output is captured silently and returned as a CommandResult instance.

---

2. Execution Modifiers

Modifiers are placed immediately after the closing parenthesis ) of a command:

Modifier	Syntax	Behavior	Python Equivalent
Strict (!)	$(cmd)!	Raises CommandError if exit code != 0	run(..., check=True)
Safe (?)	$(cmd)?	Suppresses errors; never raises exceptions on failure	run(..., suppress_errors=True)
Background (&)	job = $(cmd) &	Spawns in background non-blockingly; returns BackgroundJob	run_bg(...)

Examples:

# 1. Strict mode: abort pipeline if build fails
try:
    $(docker build -t app:latest .)!
except Exception as err:
    print(f"Build failed with exit code: {err.result.exit_code}")

# 2. Safe mode: probe an endpoint or optional service without try/except
probe = $(curl -sSf http://localhost:8080/health)?
if probe:
    print("Service is healthy!")
else:
    print(f"Service offline (exit code: {probe.exit_code})")

# 3. Background jobs: run long-running tasks concurrently (&)
job = $(mvn clean package) &
print("Maven build started in background...")
result = job.wait()
print(f"Build finished with code: {result.exit_code}")

# 3.1 Waiting for multiple parallel background jobs with wait_all(...)
j_api = $(deploy-service api) &
j_web = $(deploy-service web) &
j_db  = $(deploy-service db) &

# Accepts variable arguments wait_all(j1, j2, ...) or a list wait_all([j1, j2, ...]):
all_results = wait_all(j_api, j_web, j_db)
for res in all_results:
    print(f"Completed: {res.command} (exit code: {res.exit_code})")

# 4. Async / Await inside coroutines
async def pull_repo(name: str):
    res = await $(git -C {name} pull origin main)
    return res.stdout

# 4.1 Running multiple async commands in parallel with asyncio.gather
async def update_all_microservices():
    repos = ["frontend", "backend", "worker"]
    # All pull commands execute concurrently:
    results = await asyncio.gather(*(pull_repo(r) for r in repos))
    print(f"Updated {len(results)} repositories simultaneously.")

---

3. Chaining Methods & Modifiers

You can chain properties and helper methods directly onto command expressions:

.json — Structured JSON Output

Automatically parses JSON standard output into a navigable DynamicObj:

pods = $(kubectl get pods -o json).json
for item in pods.items:
    print(f"Pod: {item.metadata.name} | Status: {item.status.phase}")
    # Supports both dot access and dict access:
    print(f"Namespace: {item['metadata']['namespace']}")

.tee — Live Console Streaming + Output Capture

Streams stdout/stderr in real-time to the console while simultaneously capturing the complete result in the variable:

# Output is displayed immediately on screen AND stored in 'test_run'
test_run = $(pytest tests/ -v).tee
if test_run.exit_code != 0:
    print("Failed test log:", test_run.stderr)

.input(...) — Feeding Stdin Data

Pipes string or binary data directly into the standard input of the subprocess:

manifest = """
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
"""
$(kubectl apply -f -).input(manifest)

.lines & .text

• .lines: Returns a list[str] of non-empty lines from stdout (stripped of trailing newlines).
• .text: Returns the trimmed stdout string (stdout.strip()).

branches = $(git branch --list).lines
first_line = $(head -n 1 file.txt).text

---

4. Interpolation & Quoting Semantics

spy provides precise rules for parameter interpolation to keep shell scripts intuitive:

Unquoted Variable / Expression Interpolation

Any Python expression enclosed in {...} is evaluated and inserted into the command string:

target_cluster = "prod-us-east-1"
$(kubectl config use-context {target_cluster})
$(az vm list --resource-group {config.resource_group})

Double Quotes ("...") — Interpolation Enabled

Double-quoted command strings expand {expression}:

name = "World"
$(echo "Hello, {name}!")
# Evaluates to: echo "Hello, World!"

Single Quotes ('...') — Strictly Literal

Single-quoted command strings preserve braces literally. No interpolation occurs inside single quotes. This is critical for shell tools like awk, regex patterns, or inline sub-scripts:

# Braces remain literal {print $1}:
$(awk '{print $1}' access.log)

# Regex stays literal:
$(grep -E '^[0-9]{4}-[0-9]{2}' server.log)

List Expansion / Splat ({*iterable})

Expands a Python list, tuple, or iterable into space-separated command-line arguments:

files = ["service.py", "models.py", "utils.py"]
$(ruff check {*files})
# Transpiles to: run_expanded("ruff", "check", *files, capture=False)

---

5. Preserved Shell Semantics

spy passes command strings to the underlying shell without interfering with native shell operators.

Pipelines (|)

Connect the standard output of one command directly to the standard input of the next:

# 1. Pipeline in statement form (streaming output directly to terminal)
$(kubectl get pods -n prod | grep -v Completed | sort)

# 2. Pipeline in expression form (captured and iterated)
failed_jobs = $(docker ps -a | grep "Exited (" | awk '{print $1}').lines
for container_id in failed_jobs:
    print(f"Removing dead container: {container_id}")
    $(docker rm {container_id})

# 3. Chaining with Python processing
build_errors = $(cargo check 2>&1 | grep "error\[E").lines
if build_errors:
    print(f"Found {len(build_errors)} compile errors:")
    for err in build_errors:
        print("  -", err)

Redirections (>, >>, <)

Direct process outputs or inputs to and from filesystem files:

# Overwrite file with stdout (>)
$(terraform output -json > tf_outputs.json)

# Append to log file (>>)
$(date >> deployment.log)
$(echo "Deployed by {user} on {branch}" >> deployment.log)

# Read input from file (<)
$(mysql -u root -p{db_pass} my_database < migration.sql)!

Subcommands ($(...))

Inner shell command substitutions are handled directly by the shell runtime:

# Create timestamped tarball using subshell date command:
$(tar -czf backup-$(date +%Y%m%d).tar.gz /var/data)

# Create git release tag from file content:
$(git tag release-$(cat VERSION))

---

6. The CommandResult Object

Captured command expressions return a CommandResult instance with rich inspection capabilities:

Attribute / Method	Type	Description
res.stdout	str	Full standard output
res.stderr	str	Full standard error
res.exit_code	int	Process exit status code (0 = success)
res.duration	float	Command execution time in seconds
res.command	str	Exact command string executed
res.lines	list[str]	List of non-empty stdout lines
res.text	str	Trimmed standard output (stdout.strip())
res.json	DynamicObj	Parsed JSON object / list
for line in res:	Iterator[str]	Iterate directly over lines in stdout
res[index]	str	Access a specific line by index
bool(res)	bool	Truthiness: True if exit_code == 0, else False

Truthiness Example:

if $(git diff --quiet):
    print("Working tree clean")
else:
    print("Uncommitted changes detected")

---

7. Built-in Context Managers

Every .spy script and module automatically has access to cd() and env() as first-class primitives without requiring any manual import statement.

with cd(path) — Directory Navigation

Changes current working directory for the duration of the with block and guarantees restoration to the previous directory upon exiting, even if an exception occurs:

# 1. Work in a specific subproject directory
with cd("services/billing"):
    $(cargo build --release)!
    $(cargo test)

# 2. Nested directory navigation
with cd("packages"):
    with cd("frontend"):
        $(npm test)
    # Automatically back in "packages"
# Automatically back in the root directory

with env(**kwargs) — Temporary Environment Variables

Sets or overrides environment variables for the duration of the with block and safely restores the original environment afterwards:

with env(AWS_DEFAULT_REGION="eu-west-1", STAGE="staging"):
    $(aws s3 ls)
    $(serverless deploy)
# AWS_DEFAULT_REGION and STAGE are restored to their original values

Combining cd() and env()

You can combine multiple context managers cleanly on a single line:

with cd("apps/backend"), env(DATABASE_URL="postgres://test:5432/db", LOG_LEVEL="DEBUG"):
    $(alembic upgrade head)!
    $(pytest -v)

---

8. Modular Architecture (.spy Imports)

You can structure large DevOps and infrastructure projects into modular files. .spy scripts can import other .spy scripts or packages natively:

# main.spy
import devops_utils
from infrastructure.cloud import deploy_cluster

status = devops_utils.get_git_status()
deploy_cluster("production")

The underlying import hook compiles .spy files into standard Python bytecode on the fly with zero disk pollution.

---

9. Interactive REPL

spy includes a dedicated interactive read-eval-print loop with instant transpilation:

# Launch interactive shell
spy repl
# or simply
spy

>>> branch = $(git branch --show-current).text
>>> branch
'main'
>>> for file in $(git ls-files):
...     if file.endswith(".spy"):
...         print("Found spy script:", file)
... 

---

Examples

The repository includes runnable .spy examples in the examples/ directory:

• examples/demo.spy: Basic overview demonstrating variable interpolation, list expansion, and status checking.
• examples/syntax_reference.spy: Comprehensive, executable reference covering every syntax construct and execution mode.
• examples/advanced_features.spy: Practical demonstration of the 8 advanced productivity features (streaming, .tee, .input(...), cd()/env(), background jobs &, safe mode ?, quote semantics).
• examples/parallel_async_jobs.spy: Concurrent process orchestration showing how to launch multiple background jobs with &, await them all with wait_all(...), and coordinate async coroutines with asyncio.gather(...).
• examples/complex_devops.spy: Advanced pipeline orchestrator demonstrating Python dataclasses, object-oriented design, dynamic JSON parsing, strict mode error handling (try/except CommandError), and splat expansion.
• examples/modular_demo.spy & examples/devops_utils.spy: Modular multi-file architecture demonstrating how a .spy file can seamlessly import reusable functions, classes, and shell workflows from another .spy file or package.

Run them directly with spy:

spy examples/demo.spy
spy examples/syntax_reference.spy
spy examples/advanced_features.spy
spy examples/parallel_async_jobs.spy
spy examples/complex_devops.spy
spy examples/modular_demo.spy

Getting Started

Prerequisites

• Python >= 3.12
• uv package manager
• Supported Platforms: Linux, macOS, and Windows (all tested in CI)

Development Setup

# Install dependencies
uv sync

# Run tests
uv run pytest

# Run CLI help
uv run pycli --help

Editor Support

Language extensions and syntax highlighting definitions are provided in the editors/ directory:

• Visual Studio Code & Antigravity IDE (editors/vscode): Full Python + embedded shell grammar, command delimiter highlighting, interpolation scoping, and snippets (cmd, cmdvar, cmdjson, cmdstrict, cmdsplat, cmdif).
  • To install in VSCode:

    Copy-Item -Recurse -Force "editors/vscode" "$env:USERPROFILE\.vscode\extensions\pycli-vscode"
    

  • To install in Antigravity IDE:

    Copy-Item -Recurse -Force "editors/vscode" "$env:USERPROFILE\.antigravity-ide\extensions\pycli-vscode"
    

• Notepad++ (editors/notepadplusplus): User Defined Language (UDL) definition for .spy files.
  • To install locally:

    Copy-Item -Force "editors/notepadplusplus/pycli.xml" "$env:APPDATA\Notepad++\userDefineLangs\"