linkinpy

linkinpy is the Python core for running YAML-described bioimage-analysis methods in isolated uv environments. It provides:

• a CLI: list, install, run, validate, schema, batch, pipeline, envs, gc, serve and gui
• spec parsing, linting (validate) and runtime validation of inputs
• per-spec environment management (locked installs, automatic recreation when the Python version changes)
• image loading, writing, and shared-memory transport helpers
• result normalization for single-output and multi-output methods, with real CSV/JSON/TIFF writers
• batch runs, spec-declared pipelines and a provenance record (linkinpy_run.json) for every run
• progress reporting and live log streaming
• a remote host and client (serve, remote list|install, run --remote) for running on another machine
• Colab helper widgets
• a Python facade API plus lower-level functions

Run commands in this document from the LinkinPy/ folder.

Development Setup

uv sync
uv run ruff check src tests
uv run pytest          # tests/test_core.py and tests/test_features.py

Optional developer commands:

uv run ruff format .
uv run python -m build
uv run pre-commit install

The same workflow is available through make:

make sync
make lint
make test
make format
make build

CLI Tutorial

List available methods:

uv run linkinpy list ../Library/nanopyx.yaml

Install a spec for the CLI interface:

uv run linkinpy install ../Library/nanopyx.yaml --interface cli

Each spec gets its own environment under ~/.linkinpy/<spec-name>/. If the spec contains how_to_cite, the install command prints that citation text after the install summary.

Run a method and save declared outputs to a folder:

uv run linkinpy run ../Library/nanopyx.yaml eSRRF \
  --output-dir outputs \
  image=input.tif \
  magnification=2

Use explicit output destinations when needed:

uv run linkinpy run ../Library/StarDist.yaml "Predict 2D Pretrained Model" \
  --output labels=outputs/labels.tif \
  image=input.tif \
  model_name=2D_versatile_fluo

Arguments are passed as name=value. For inputs declared as np.ndarray, the CLI accepts image paths and .npy files.

CLI Reference

Command	Purpose
linkinpy list SPEC	List the methods a spec declares
linkinpy install SPEC [--interface I]	Create or update the spec's environment (prints how_to_cite)
linkinpy run SPEC METHOD [name=value ...]	Run a method (--output-dir, --output name=path, or --remote URL to run on a host)
linkinpy validate SPEC [--strict]	Lint a spec; non-zero exit on errors (or warnings with --strict)
linkinpy schema	Print the JSON Schema for spec files
linkinpy batch SPEC METHOD --input-dir D --output-dir O	Run one method over many images
linkinpy pipeline list SPEC / run SPEC NAME	List or run the spec's pipelines
linkinpy envs list / refresh NAME / remove NAME	Inspect and manage environments
linkinpy gc [--dry-run] [--older-than DAYS]	Remove stale temp files and orphaned shared memory
linkinpy serve [--port N]	Serve installed environments to remote clients (the "host")
linkinpy remote list / install SPEC	List what a host has installed, or install a spec on it
linkinpy gui SPEC METHOD	Open a generated Qt (magicgui) window for one method

SPEC is a local path or an https:// URL. Every command has --help.

Validate and edit specs with completion

uv run linkinpy validate ../Library/StarDist.yaml            # 0 error(s), 0 warning(s)
uv run linkinpy validate --strict my_spec.yaml               # warnings fail too (used in CI)
uv run linkinpy schema > spec.schema.json

Add this as the first line of a spec to get completion in editors that use yaml-language-server:

# yaml-language-server: $schema=https://raw.githubusercontent.com/HenriquesLab/LinkinPy/main/docs/spec.schema.json

validate reports errors (duplicate names, min > max, a default outside choices, unsafe output names, unknown pipeline steps...) and warnings (unknown type/role/format, missing metadata). At run time the declared type, min, max and choices are enforced before the method is called, so a bad value fails fast with a clear message:

linkinpy.runner.RunnerError: Input 'gain' must be <= 10, got 99.0.

Batch: one method over a folder

uv run linkinpy batch ../Library/StarDist.yaml "Predict 2D Pretrained Model" \
  --input-dir raw/ --output-dir results/ --recursive --pattern "*.tif" --jobs 2 \
  model_name=2D_versatile_fluo

[1/3] ok      a.tif (4.2s)
[2/3] ok      b.tif (3.9s)
[3/3] failed  c.tif (0.8s) ValueError: ...
2 ok, 1 failed, summary in results/batch_summary.csv

Each image gets its own folder (results/<relative path>/<image stem>/) with its outputs and linkinpy_run.json. The environment is installed once up front, one failing image never stops the batch, --skip-existing resumes an interrupted run, and the exit code is 1 if anything failed. --image-arg names the image input when a method has several.

Pipelines: chain methods inside one spec

pipelines:
  denoise_then_segment:
    steps:
      - id: denoise
        method: Denoise
        inputs: {image: "$input.image", strength: 2}
      - id: segment
        method: Segment
        inputs: {image: "$denoise.result"}

uv run linkinpy pipeline list my_spec.yaml
uv run linkinpy pipeline run my_spec.yaml denoise_then_segment --output-dir out/ image=cells.tif

$input.<name> is a pipeline argument and $<step id>.<output name> is an earlier step's output. Step outputs land in out/<step id>/. See METHOD_AUTHORING.md.

Environments and cleanup

uv run linkinpy envs list                 # name, Python version, size, stored spec
uv run linkinpy envs refresh stardist     # reinstall packages (e.g. after upgrading LinkinPy)
uv run linkinpy envs remove stardist
uv run linkinpy gc --dry-run              # preview; drop --dry-run to delete

gc removes *.part downloads, old linkinpy-* temp folders and, where the platform lists them (Linux /dev/shm), orphaned shared-memory segments.

Provenance

Runs that write outputs also write linkinpy_run.json next to them:

{
  "callable": "Blur",
  "interface": "cli",
  "spec": {"display_name": "E2E Demo", "version": "1", "sha256": "6d8c63db..."},
  "environment": {"python": "3.12.13", "packages": {"linkinpy": "0.0.8"}},
  "arguments": {"gain": 4.0, "image": "/data/a.tif"},
  "outputs": [{"name": "result", "path": "out/result.tif", "sha256": "afba6106...", "bytes": 1234}],
  "started": "2026-10-16T12:00:00+00:00", "finished": "2026-10-16T12:00:02+00:00"
}

Progress

A method reports progress with linkinpy.progress(fraction, message). linkinpy run prints it as [ 50%] halfway on a terminal and as a raw JSON line when piped (hosts read that form).

Running on another machine

1. On the host machine (the GPU box) start the server. It needs only linkinpy; the option you will normally set is the port:

uv run linkinpy serve --port 8765        # prints a bearer token; binds to localhost by default

2. From your laptop, tunnel the port, then connect, install and run. None of this needs the tool's packages (TensorFlow, PyTorch...) on your laptop:

ssh -N -L 8765:localhost:8765 gpu-host &
export LINKINPY_REMOTE=http://localhost:8765
export LINKINPY_TOKEN=<token printed by serve>

uv run linkinpy remote list                              # what the host has installed
uv run linkinpy remote install StarDist.yaml             # install a Library spec ON THE HOST
uv run linkinpy remote install my_tool.yaml              # ...or a local file (uploaded) or an https URL

uv run linkinpy run ../Library/StarDist.yaml "Predict 2D Pretrained Model" \
  --remote http://localhost:8765 --output-dir results/ image=input.tif model_name=2D_versatile_fluo

Image inputs are uploaded and outputs are downloaded. The server only runs specs installed on it; installing is done by the host, with a streamed log. For use beyond an SSH tunnel pass --certfile and --keyfile for TLS; the token is the only access control, so treat it like a password.

napari, Fiji and QuPath talk to the same server from a separate Remote submenu with three entries: Connect to host, Install on host and Run on host. The run entry lists only the specs and methods the host reports as installed, and builds its parameter dialog from the host's description, so nothing has to be installed locally. See each plugin's README.

Without --output-dir

If you give neither --output-dir nor --output, the CLI writes outputs to a temporary folder and prints its path. (Image outputs cannot be returned over a plain pipe, and an unreleased shared-memory segment would leak.)

Environment variables

Variable	Meaning
LINKINPY_ENV_HOME	Where specs and environments live (default ~/.linkinpy)
LINKINPY_REMOTE	Default server URL for run --remote
LINKINPY_TOKEN	Bearer token for serve, run --remote and remote ...

Python API Tutorial

The facade

from linkinpy import LinkinPy

lp = LinkinPy("../Library/nanopyx.yaml")
print(lp.list_callables())
print(lp.validate())                       # [] when the spec has no errors or warnings
lp.install(interface="cli")
stdout = lp.run(
    "eSRRF",
    {"image": "input.tif", "magnification": 2},
    output_dir="outputs",
)

run returns the worker's output; turn it into result objects with parse_run_result:

from linkinpy import parse_run_result

for item in parse_run_result(stdout):
    print(item.spec.name, item.path)       # e.g. "result outputs/result.tif"

Batch, pipelines and streaming

from linkinpy import LinkinPy, load_spec, parse_progress_line, run_spec_callable_in_environment

lp = LinkinPy("my_spec.yaml")

batch = lp.batch("Blur", input_dir="raw/", output_dir="results/", arguments={"gain": 2})
print(len(batch.ok), "ok,", len(batch.failed), "failed")        # BatchItem: input_path, status, error, outputs

pipeline = lp.pipeline("denoise_then_segment", {"image": "cells.tif"}, output_dir="out/")
print(pipeline.final["labels"].path)                             # outputs of the last step

def on_line(line):                                               # called for every output line, live
    progress = parse_progress_line(line)                         # (fraction | None, message) or None
    print(progress if progress else line)

spec = load_spec("my_spec.yaml")
run_spec_callable_in_environment(spec, "Blur", {"image": "a.tif"}, output_dir="out/", on_line=on_line)

Remote runs

from linkinpy import run_remote

outputs = run_remote(
    "http://localhost:8765", token, "stardist",           # server URL, token, installed spec name
    method="Predict 2D Pretrained Model",
    arguments={"image": "cells.tif", "model_name": "2D_versatile_fluo"},
    output_dir="results/",
    on_line=print,
)
print(outputs["labels"])                                  # local path of the downloaded output

RemoteClient(url, token) exposes the individual steps: info() (server version), specs() (everything installed on the host, with full input/output descriptions), install(...), upload(), submit(), wait() and download():

from linkinpy import RemoteClient

client = RemoteClient("http://localhost:8765", token)
print(client.info())                                      # {'ok': True, 'linkinpy': '0.0.8'}
job = client.install(spec_name="StarDist.yaml", interface="cli")   # or spec_yaml=..., spec_url=...
client.wait(job, on_line=print)                           # streams the install log
print([spec["name"] for spec in client.specs()])

Results in memory (display-only methods)

from linkinpy import load_spec, run_for_display

spec = load_spec("my_spec.yaml")                      # a method with output: {napari: [display]}
results = run_for_display(spec, "Segment", {"image": image_array}, interface="napari")
labels = results["labels"].value                      # NumPy array, copied out of shared memory
print(spec.callables["Segment"].actions_for("napari"))   # (display, io) == (True, False)

Nothing is written to disk; only outputs with display: true are returned. See METHOD_AUTHORING.md and the host protocol in CONTRIBUTING.md.

Reporting progress from a method

from linkinpy import progress

def run(image, n_tiles: int = 8):
    for i in range(n_tiles):
        ...
        progress((i + 1) / n_tiles, f"tile {i + 1}/{n_tiles}")

Errors

A failure inside the spec's environment (bad argument, exception in the method) raises LinkinPyEnvironmentError carrying the worker's message, e.g. ... RunnerError: Input 'gain' must be <= 10, got 99.0.. Calling a method in-process (run_spec_callable(..., ensure_environment=False)) raises RunnerError directly. Spec problems raise SpecError.

Lower-level APIs

• load_spec / parse_spec, validate_spec (returns Issue objects with level, where, message)
• EnvironmentManager, list_environments, refresh_environment, remove_environment, collect_garbage
• run_spec_callable, run_spec_callable_in_environment, run_for_display, parse_run_result
• run_batch, run_pipeline, run_remote, RemoteClient
• read_image / write_image, create_shared_array / attach_shared_array
• progress, parse_progress_line

Colab Tutorial

Install LinkinPy in a notebook:

!pip install linkinpy

Mount Google Drive:

from linkinpy import mount_google_drive

mount_google_drive()

Install a selected spec:

from linkinpy import install_colab_spec

install_colab_spec("/content/LinkinPy/Library")

Run a method from installed specs:

from linkinpy import run_installed_colab_method

run_installed_colab_method(
    output_dir="/content/drive/MyDrive/LinkinPy/outputs"
)

Colab displays image and labels outputs when possible and prints metadata/path outputs in the notebook output area.

Spec Contract

Specs are YAML files. Each method declares inputs, outputs, and output.

display_name: Example Package
python_version: "3.12"
how_to_cite: "Author A, Author B. Example Package. Journal, year."

install:
  python_packages:
    - name: example-package

segment:
  callable: example_package.segment
  inputs:
    - name: image
      display_name: Input Image
      type: np.ndarray
      role: image
      required: true
  outputs:
    - name: labels
      display_name: Label Image
      type: np.ndarray
      role: labels
      format: tif
      display: true
  output:
    cli: [io]
    colab: [display, io]
    napari: [display, io]
    imagej: [display, io]
    qupath: [display, io]

outputs is the return-value contract. Tuple/list returns map by output order. Dict returns map by output name.

output: says, per interface, what happens to the results: [display] shows them in the host and writes nothing to disk (images through shared memory, tables and shapes inline, no output folder), [io] saves them, [display, io] does both. If shared memory cannot be used, display-only runs fall back to temporary files that are deleted after display.

Output role tells hosts how to present a result: image, labels, scalar, metadata, text, path, folder, plus table (CSV), points (CSV of y,x[,...] rows) and geojson (GeoJSON FeatureCollection). A spec can also declare pipelines:, and a method can embed its code with code: instead of a package. All of this, with examples, is in METHOD_AUTHORING.md.

More Documentation

• GETTING_STARTED.md for contributor setup
• METHOD_AUTHORING.md for package authors
• CONTRIBUTING.md for the contribution workflow
• CHANGELOG.md for what changed
• Repository tutorials for all interfaces, with examples