Skip to main content

Picsellia Pipelines CLI

The Picsellia Pipelines CLI lets you quickly create, test, dockerize, deploy, and manage custom processing or training pipelines.

How it works

A pipeline is simply:

  • a Python function (pipeline.py)

  • calling steps (steps.py)

  • configured via run_config.toml

  • run locally or on Picsellia infrastructure

Workflow at a glance

  1. Init → generate project template

  2. Customize → implement steps & parameters

  3. Test → run locally

  4. Smoke Test → validate Docker image

  5. Deploy — Publish Your Pipeline to Picsellia → push & register pipeline

1. Installation

Show installation instructions
uv pip install picsellia-pipelines-cli

With Poetry:

poetry add picsellia-pipelines-cli

Check installation:

pxl-pipeline --help

2. Authentication

Objective: Use the same Picsellia user and environment across all commands

pxl-pipeline login
Show detailed explanation

This stores your:

  • organization
  • environment (PROD / BETA / DEV / CUSTOM)
  • API token
  • optional custom base URL

Other helpful commands:

pxl-pipeline whoami   # show active context
pxl-pipeline switch   # change organization/environment
pxl-pipeline logout   # clear active context

3. Init — Create a new pipeline

Objective: Generate a ready-to-use pipeline project folder with all required template files

pxl-pipeline init <pipeline_name> --type [training|processing] --template <template_name>
Show detailed explanation

Examples:

pxl-pipeline init yolov8 --type training --template yolov8
pxl-pipeline init resize-images --type processing --template dataset_version_creation

This generates:

  • a single entrypoint: pipeline.py

  • config.toml (metadata + execution parameters)

  • Dockerfile and .dockerignore

  • a consistent folder structure:

my-pipeline/
├── pipeline.py
├── steps.py
├── utils/
│   ├── parameters.py
├── config.toml
├── Dockerfile
├── .dockerignore
├── runs/
│   └── run_config.toml   # template for test/smoke-test/launch
└── pyproject.toml

You're now ready to implement your custom logic.

4. Customize your pipeline — Add steps & parameters

Objective: Adapt the pipeline template to your specific use case

After running init, your pipeline project is generated with a default structure:

  • pipeline.py — your entrypoint

  • steps.py (a default process step) — implement processing or training steps

  • utils/parameters.py — define your pipeline parameters

👉 In most cases, if you chose the right template, you only need to modify the existing process step to implement your use case.

You do not need to redesign the whole pipeline unless your logic is more advanced.

  1. Start by editing the existing process step

    This step already exposes the correct inputs, outputs, and parameters expected by Picsellia.

  2. Only add new steps if needed

    If your pipeline requires additional logic (pre-processing, post-processing, custom validation, chaining operations, etc.), you can:

    • modify the existing step
    • replace it entirely
    • or add new steps and compose them in pipeline.py

Working with Steps

Show detailed explanation

Steps are Python functions decorated with @step. You can define them in steps.py and call them freely inside pipeline.py.

Example (steps.py):

from picsellia_pipeline.core import step

@step
def process(dataset_input, dataset_output):
    # your processing logic here
    dataset_output["images"] = [img.upper() for img in dataset_input["images"]]
    return dataset_output

Example pipeline (pipeline.py):

from steps import process
from picsellia_pipeline.core import pipeline, step

@pipeline
def dataset_version_creation_pipeline():
    dataset_collection = load_coco_datasets()
    dataset_collection["output"] = process(
        dataset_collection["input"], dataset_collection["output"]
    )
    upload_full_dataset(dataset_collection["output"], use_id=False)
    return dataset_collection

Adding Parameters

Show detailed explanation

Define pipeline parameters in utils/parameters.py.

All parameters declared here are:

  • automatically detected by the CLI

  • injected at runtime

  • uploaded to Picsellia during deploy or sync

Example:

from picsellia.types.schemas import LogDataType
from picsellia_cv_engine.core.parameters import Parameters

class ProcessingParameters(Parameters):
    def __init__(self, log_data: LogDataType):
        super().__init__(log_data=log_data)
        self.datalake = self.extract_parameter(["datalake"], expected_type=str, default="default")
        self.data_tag = self.extract_parameter(["data_tag"], expected_type=str, default="processed")

Each parameter requires:

  • a key → used as the parameter name on the Picsellia platform

  • an expected type → str, int, float, etc.

  • a default value → mandatory for parameter registration

Once parameters are defined, you can reference them directly in your step logic and override their values in run_config.toml.

5. Test — Run your pipeline locally

Objective: Ensure your Python code works exactly as expected, using real Picsellia objects

pxl-pipeline test <pipeline_name> --run-config-file <path>
Show detailed explanation

⚠️ Important

Even though the pipeline runs locally, all datasets, experiments, and outputs are created and updated on the Picsellia platform.

This command:

  • runs the pipeline locally in the virtual env (.venv/)
  • loads the configuration from your run_config.toml
  • interacts with real Picsellia objects
  • uploads results to the platform exactly like a real run
  • guarantees your step logic and parameters behave correctly

A template run config is generated automatically at:

<pipeline_name>/runs/run_config.toml

You simply need to fill it with your dataset/model IDs, parameters, or metadata.

Example (dataset version creation processing):

override_outputs = true

[job]
type = "DATASET_VERSION_CREATION"

[input.dataset_version]
id = ""

[output.dataset_version]
name = "test_my_pipeline"

[parameters]
datalake = "default"
data_tag = "processed"

Once the file is filled:

pxl-pipeline test my-pipeline \
  --run-config-file my-pipeline/runs/run_config.toml

6. Smoke Test — Validate the Docker runtime

Objective: Ensure your Dockerfile, dependencies, imports, paths, and runtime fully work before deployment

pxl-pipeline smoke-test <pipeline_name> --run-config-file <path>
Show detailed explanation

This command:

  1. Builds the Docker image
  2. Runs the pipeline inside the container (not Python locally)
  3. Uses the same run_config.toml as the test command
  4. Updates real objects/results on Picsellia

It is your final validation step before deployment.

A successful smoke test strongly indicates that the pipeline will run properly on Picsellia’s infrastructure.

7. Deploy — Publish your pipeline to Picsellia

Objective: Build, version, push the Docker image, and register/update the pipeline in your organization

pxl-pipeline deploy <pipeline_name>
Show detailed explanation

This command:

  • builds the Docker image
  • pushes it to your configured registry
  • versions the image
  • creates or updates the Picsellia processing or training asset
  • ensures the pipeline is ready to be launched from the UI

For training pipelines, define either one [model_version] or several [[model_versions]] rows — not both (the CLI will reject ambiguous configs).

Single target:

[model_version]
origin_name = "MyModel"
name = "v1"
framework = "ONNX"
inference_type = "OBJECT_DETECTION"

Multiple targets (each row must include all fields):

[[model_versions]]
origin_name = "MyModel"
name = "v1"
framework = "ONNX"
inference_type = "OBJECT_DETECTION"

[[model_versions]]
origin_name = "MyModel"
name = "v2"
framework = "ONNX"
inference_type = "OBJECT_DETECTION"

deploy propagates Docker image/tag and default parameters to every listed model version.

After deployment, the pipeline becomes usable by your team in the Picsellia interface.

Launch — Run your pipeline on Picsellia’s infrastructure

(Optional)

Objective: Trigger a real Picsellia job (not local), using the same run_config.toml

pxl-pipeline launch <pipeline_name> --run-config-file <path>
Show detailed explanation

Launch behaves like:

  • launching a processing job on a dataset
  • or launching a training experiment
  • without manually creating an experiment or job in the UI

The run_config.toml defines:

  • the dataset/model input
  • the output dataset or experiment name
  • the pipeline parameters

This is equivalent to triggering an actual job from the Picsellia UI.

Sync — Synchronize local parameters with Picsellia

(Optional)

Objective: Update parameters stored on Picsellia to match your local

pxl-pipeline sync <pipeline_name>
Show detailed explanation

For processing pipelines, this syncs:

  • default parameter values
  • parameter schema / types

Sync is usually unnecessary if you run:

pxl-pipeline deploy

because deploy already updates the parameter definition on the platform.

Training sync is not yet implemented.

Tips

  • Use --output-dir during init to generate the pipeline elsewhere
  • Virtual environments are created in <pipeline>/.venv
  • You can edit config.toml at any time (metadata, entrypoints, dependencies)
  • Always run test → smoke-test → deploy for a clean workflow
  • A successful smoke test almost guarantees a successful production run

Made with ❤️ by the Picsellia team.

Metadata

Release files for picsellia-pipelines-cli 0.7.5

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for picsellia-pipelines-cli 0.7.5
File Size Uploaded
picsellia_pipelines_cli-0.7.5.tar.gz 72.3 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for picsellia-pipelines-cli 0.7.5
File Interpreter ABI Platform
picsellia_pipelines_cli-0.7.5-py3-none-any.whl Python 3 none any Details

Total release size: 169.8 kB

Release files / picsellia_pipelines_cli-0.7.5.tar.gz

Download URL picsellia_pipelines_cli-0.7.5.tar.gz
Size 72.3 kB
Tags Source
SHA-256 checksum
How to use checksums
1671abf9596a8bd3955b1645f849b9e56abde73f7fe63e1234b9124ce54a5b6a
BLAKE2b-256 checksum
How to use checksums
d927aa14af4129145edb35521b70fd05032605cb2ca8d30eb030ae1e91cad8e0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.12.10 {"installer":{"name":"uv","version":"0.12.10","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / picsellia_pipelines_cli-0.7.5-py3-none-any.whl

Download URL picsellia_pipelines_cli-0.7.5-py3-none-any.whl
Size 97.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
408c50bac9d1cd4b52d894c7693790b16933375268e67888bf87eb24a3d04a51
BLAKE2b-256 checksum
How to use checksums
616b421093e543a2e23ac3f33b5bd0a2da38639eb5387dfac773dd591b794b45
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.12.10 {"installer":{"name":"uv","version":"0.12.10","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release history Release notifications | RSS feed

0.7.6

2 release files

This release

0.7.5 This release

2 release files

0.7.4

2 release files

0.7.3

2 release files

0.7.2

2 release files

0.7.1

2 release files

0.7.0

2 release files

0.6.0

2 release files

0.5.0

2 release files

0.4.0

2 release files

0.3.2

2 release files

0.3.1

2 release files

0.3.0

2 release files

0.2.0

2 release files

0.1.2

2 release files

0.1.1

2 release files

0.1.0

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page