griip-sdk

Hardware control and orchestration for bin picking, by Vention.

The SDK owns the robot, gripper, perception, and planning. You own the loop: when to pick, where the part goes, and what to do on failure.

Install

pip install griip-sdk

Python 3.10. Needs a running mmai-griip-api. From outside the Vention monorepo, use gRPC mode.

Wheels publish to public PyPI from master via the monorepo nx release pipeline. Pull-request CI only dry-runs. Cells that still keep private extras under /opt/vention/wheels/ can add --find-links /opt/vention/wheels/.

Testing a pull request

Every pull request that touches the SDK publishes a dev build to its own PyPI project, so you can try a branch before it merges:

pip uninstall -y griip-sdk          # both own the `griip_sdk` import package
pip install griip-sdk-dev==<version>

The version is on the pull request's CI summary, as <next release>.dev<run number>. pip show griip-sdk-dev names the commit it was built from. Dev builds depend on released siblings, so a branch that also changes griip-core is only half-covered by one. Never install one on a production cell.

Quickstart

from griip_sdk import build_pick_only_manager

DROP_JOINTS = [-1.5, -1.2, 1.0, -1.4, -1.5, 0.0]   # your drop pose

with build_pick_only_manager("./pick.yaml", part_id="pcb") as manager:
    pick = manager.pick(max_attempts=3)
    if pick.succeeded:
        manager.start_pick_generation()                 # perception runs while you move
        manager.collision_free_move_to_joints(DROP_JOINTS, object_in_tcp=pick.obj_in_tcp)
        manager.release()

Two entry points, same setup under the hood:

• build_pick_only_manager for single-cycle pick-only work (above).
• build_bin_picking_manager for a continuous bin-pick + placement loop; you pass a handle_placement callback and call manager.run().

Both return a context manager that wires up hardware and tears it down on exit.

Handling failures

manager.pick() always returns a PickResult. On failure, failure_reasons holds one reason per attempt in order. failure_reasons[-1] is the last attempt; more than one distinct reason means the cell is failing for shifting reasons.

PickFailureReason	Meaning	Typical response
NO_PARTS_DETECTED	Perception saw no objects.	Request refill.
NO_GRASPABLE_PARTS	Parts visible but none reachable.	Shake / reorient.
GRIP_CHECK_FAILED	Scooped but nothing in hand.	Retry next loop.
MOTION_FAILED	Collision or scoop plan/exec failure.	Alert operator.
PERCEPTION_FAILED	Picking stream died mid-call (gRPC error).	Backoff + retry, or page ops.
PLACEMENT_FAILED	Pick OK, place leg failed (placement mode).	Cell-specific.

pick.attempts_made and pick.detail are for logs and metrics, not control flow. Hardware faults (robot or gripper driver) don't map to a reason: they escape as exceptions, which is your "stop the cell" signal.

A loop that acts on each reason:

from griip_sdk import build_pick_only_manager, PickFailureReason

with build_pick_only_manager("./pick.yaml", part_id="pcb") as manager:
    while True:
        pick = manager.pick(max_attempts=3)

        if pick.succeeded:
            manager.start_pick_generation()
            manager.collision_free_move_to_joints(DROP_JOINTS, object_in_tcp=pick.obj_in_tcp)
            manager.release()
            continue

        if len(set(pick.failure_reasons)) > 1:
            alert_operator(f"cell unstable: {pick.failure_reasons}")
            break

        match pick.failure_reasons[-1]:
            case PickFailureReason.NO_PARTS_DETECTED:
                request_refill()
            case PickFailureReason.NO_GRASPABLE_PARTS:
                shake_bin()
            case PickFailureReason.GRIP_CHECK_FAILED:
                pass                       # SDK already retried; try again next loop
            case PickFailureReason.MOTION_FAILED | PickFailureReason.PERCEPTION_FAILED:
                alert_operator(pick.detail)
                break

Manager methods

Method	Effect
pick(max_attempts=3)	Pick only. Returns a PickResult; robot stays at the prep pose on success.
pick_and_place_once(max_pick_attempts=3, target_pose=...)	Pick, then place when target_pose is given.
start_pick_generation()	Move to the capture pose and run perception. The camera grab is synchronous; perception runs async.
collision_free_move_to_joints(target, object_in_tcp=None, ...)	Collision-aware joint move. Pass object_in_tcp=pick.obj_in_tcp while carrying a part.
release()	Open the gripper and clear object_in_hand.
set_capture_joints(joints)	Change where perception captures from, next capture on.
set_part(part_id)	Swap the active part mid-session. Re-sends Initialize over the existing channel; no hardware or planner re-init.

pick.yaml

The cell YAML points the runner at the robot, gripper, camera, perception backends, and the cell's collision geometry. It does not hold drop poses or loop policy. That is your application code.

Bring your own gripper

Both builders take an optional gripper=. When set, the SDK uses your client instead of building one from the YAML.

Use the SDK's Robotiq 2F-140 directly, so a bench consumer keeps its hardware constants out of the cell YAML:

from griip_sdk import build_pick_only_manager
from griip_sdk.hardware.robotiq_2f140_gripper import Robotiq2F140Gripper

gripper = Robotiq2F140Gripper(serial_port="/tmp/ttyUR", baudrate=115200, modbus_slave_id=9, grip_force=20, release_width=800)

with build_pick_only_manager("./pick.yaml", part_id="pcb", gripper=gripper) as manager:
    ...

Or subclass BaseGripper for a gripper the SDK doesn't ship (a Modbus / EtherNet-IP unit, or an in-memory fake for tests):

from griip_sdk import BaseGripper, GripperMoveResult, build_pick_only_manager

class MyEthernetIPGripper(BaseGripper):
    def get_width(self) -> int: ...
    def get_current_draw(self) -> int: ...
    def is_in_safety_stop(self) -> bool: ...
    def reset_safety(self) -> None: ...
    def reconnect(self) -> None: ...
    def stop(self) -> None: ...
    def close_gripper(self, force_value=400, block=False, timeout=10.0) -> GripperMoveResult: ...
    def open_gripper(self, force_value=400, block=False) -> GripperMoveResult: ...
    def move_gripper(self, width_value, force_value=400, block=False, timeout=None) -> GripperMoveResult: ...

with build_pick_only_manager("./pick.yaml", part_id="pcb", gripper=MyEthernetIPGripper(...)) as manager:
    ...

Two things to know:

• The YAML tool.gripper block still drives manager state (closed_gripper_width, expected_grip_width_range, grip_check_width_margin, min_grip_width_for_current_part). Only the hardware-construction fields (serial_port, baudrate, modbus_slave_id, force, release_width) are ignored when you inject a gripper. The SDK logs this once at INFO.
• The SDK does no gripper-side teardown. If your gripper holds a connection (TCP or serial), close it yourself.

Bring your own camera (vision ABC)

Both builders take an optional vision_backend=. The SDK camera path talks to the vention.vision.v1 Image/Source service ABCs, so the same CameraManager drives the legacy Luxonis camera and the native gRPC cameras (vvis, Leopard) behind one interface. When vision_backend is omitted (or in mock mode) the SDK builds whichever backend the cell YAML camera.backend field selects.

The easiest route is config-only — pick the backend in the cell YAML:

camera:
  backend: "grpc"              # "mmai" (default, legacy Luxonis HTTP) or "grpc"
  target: "127.0.0.1:41051"    # vention.vision gRPC service
  sensor: "full_frame_left"    # primary source id reported by ListSources
  right_source_id: "full_frame_right"
  source_metadata: # only keys the service omits; service values always win
    camera_vendor: "leopard"
    mx_id: "LI-123456"

or build it in code with the same dispatch:

from griip_sdk import build_bin_picking_manager
from griip_sdk.hardware.grpc_vision_backend import build_vision_backend

vision_backend = build_vision_backend("./cell.yaml")  # Mmai or gRPC per camera.backend
with build_bin_picking_manager("./cell.yaml", part_id="pcb", handle_placement=..., vision_backend=vision_backend) as manager:
    ...

GrpcVisionServiceBackend composes the wheel's create_image_service_backend / create_source_service_backend gRPC backends so one object satisfies both ABC seams; build_mmai_vision_backend still builds the Luxonis adapter explicitly. Intrinsics, resolution, and stereo baseline are resolved from the backend's list_sources, so each camera reports its own calibration through the same seam — native services own their calibration (no calibio upload), and source_metadata must carry baseline / camera_vendor / mx_id (from the service or the YAML overlay) or calibration resolution fails loudly.

The camera layer imports the vention.vision.v1 protos and vision_service_abc ABCs, so it needs vention-firmware-grpc-client>=1.3.0.

Swapping parts mid-session

with build_pick_only_manager("./pick.yaml", part_id="pcb_rev_a") as manager:
    for _ in range(5):
        manager.pick(max_attempts=3)

    manager.set_part("pcb_rev_b")     # ValueError if part_id isn't in pick.yaml

    for _ in range(5):
        manager.pick(max_attempts=3)

Only works if both parts use the same gripper. If they don't, build a fresh manager for each.

Manual lifecycle (without with)

The with form is crash-safe and recommended. For a long-running daemon that needs a global-style handle, drive the lifecycle yourself:

_ctx = build_pick_only_manager("./pick.yaml", part_id="pcb")
manager = _ctx.__enter__()
try:
    manager.pick(max_attempts=3)
    # manager is usable anywhere in the process
finally:
    _ctx.__exit__(None, None, None)   # required: closes gRPC + hardware

This is an escape hatch. Skip __exit__ and you leak the gRPC channel and hardware connections.

Grasp targets, your motion

For a cell where the application moves the robot itself. The SDK takes and stores pictures, runs perception on a stored picture when asked, and keeps one queue of ranked grasp poses. It never commands motion.

from griip_sdk import build_grasp_provider

with build_grasp_provider("pick.yaml", part_id="smallest_splice") as sdk:
    # First round: one picture of each pallet, then start on the left.
    move_robot_to(LEFT_CAPTURE_POSE)     # your motion
    sdk.capture("left")                  # take a picture now, store it as "left"
    move_robot_to(RIGHT_CAPTURE_POSE)
    sdk.capture("right")                 # take a picture now, store it as "right"
    sdk.process("left")                  # start perception on the "left" picture, in the background

    while True:
        # Left pallet
        target = sdk.next_target()       # wait for the "left" run, pop its best pose; None = nothing graspable
        sdk.process("right")             # start perception on the "right" picture; it runs during the steps below
        picked_left = target is not None
        if picked_left:
            pose = target.flange_pose_mm_deg   # [x, y, z, rx, ry, rz], mm and degrees, as on the pendant
            move_robot_to(approach(pose))
            move_robot_to(pose)
            close_gripper()
            move_robot_to(approach(pose))
            sdk.capture("left")          # new picture of the left pallet for the next round, taken now
            move_robot_to(PLACE_POSE)
            open_gripper()

        # Right pallet
        target = sdk.next_target()       # wait for the "right" run, pop its best pose
        sdk.process("left")              # start perception on the new "left" picture; runs while we pick right
        picked_right = target is not None
        if picked_right:
            pose = target.flange_pose_mm_deg
            move_robot_to(approach(pose))
            move_robot_to(pose)
            close_gripper()
            move_robot_to(approach(pose))
            sdk.capture("right")         # new picture of the right pallet for the next round
            move_robot_to(PLACE_POSE)
            open_gripper()

        if not picked_left and not picked_right:
            break                        # both pallets came back empty

Five verbs. capture(tag) stores a picture with the robot pose at that moment, so park the robot first; the tag is any string you choose, one stored picture per tag. captures() lists the stored pictures, delete(tag) removes one. process(tag) runs perception on a stored picture in the background, one image at a time. next_target(timeout=None) waits for in-flight runs and pops the best pose; it returns None when the last picture had nothing graspable, when the run failed (captures() carries the state and the reason), or on timeout.

The results of a run replace the queue, so pop before starting the next run. Re-capturing a tag replaces its picture, and a run still going for the old picture is dropped.

Each GraspTarget answers what a motion-owning app needs to know:

• capture_tag says which picture, and therefore which pallet, it came from.
• Several come back per picture, ranked by priority, at most picking.max_detections × picking.max_grasps_per_object.
• flange_pose and tcp_pose are the same target in the world / robot base frame, the frame the robot reports (the camera pose is built from it and the hand-eye calibration). flange_pose = tcp_pose · X_tcp_flg⁻¹, where X_tcp_flg is the translation tool.tcp_offset, returned as tcp_offset.
• flange_pose_mm_deg and tcp_pose_mm_deg are the same poses as [x, y, z, rx, ry, rz]: millimetres, and degrees of roll, pitch, yaw about X, Y then Z, the convention the pendant shows. The SE3 forms stay in metres for code that composes transforms.

Same pick.yaml as a self-driving cell: part, grasps, tool.tcp_offset, camera, hand-eye. No home_bin* poses are needed; the capture pose is whatever the robot reports. The robot must be registered in VRMCS, where reachability is checked. No gripper is built or actuated.

Only if the pictures show a container with walls: capture(tag, bin_number=N) attaches the bin_N_* cuboids from cuboids.json to the request so grasps through a wall or below the floor are rejected. N must be listed in runtime.bins.available. It is not a taught pose. Open pallets need none of this.

Calibration

The SDK exposes the two calibration primitives — hand-eye and operator-driven environment scanning. The cell YAML is the source of truth for the calibration starting pose (positions.calibration_joints); commissioning authors it, the SDK reads it, and the operator app never edits it at runtime.

The simplest hand-eye run — build the app, call the verb:

from griip_sdk import build_calibration_app

with build_calibration_app("cell.yaml") as cal:
    x_cam_flg = cal.run_hand_eye_calibration()

The SDK plans to cell.positions.calibration_joints, runs the bundled sphere routine, returns the robot to that start pose, and persists the result. The fuller form below adds progress reporting and the operator-driven scan session:

from pathlib import Path

from griip_sdk import build_calibration_app

with build_calibration_app(Path("cell.yaml")) as cal:
    # Hand-eye: the SDK plans to cell.positions.calibration_joints, runs the
    # bundled sphere routine, then returns the robot to the start pose.
    # Override `routine=` only if your cell geometry truly needs a non-default recipe.
    x_cam_flg = cal.run_hand_eye_calibration(
        on_progress=lambda pct, status, msg: print(pct, status, msg),
        is_cancelled=lambda: False,
    )
    cal.last_hand_eye_calibration_timestamp()       # datetime | None

    # Environment scanning: operator-driven session. start_environment_session
    # switches the robot into freedrive; take_environment_image grabs one
    # frame at the current pose; end_environment_session restores normal
    # operation and persists the entry.
    cal.start_environment_session()
    for _ in range(num_frames_the_app_wants):
        # operator manually repositions the robot between captures
        cal.take_environment_image()
    sequence_dir = cal.end_environment_session()
    cal.last_environment_scan_timestamp()           # datetime | None

If cell.positions.calibration_joints is unset, run_hand_eye_calibration raises griip_sdk.ConfigError with a message naming the missing field and the config file. Surface this to the operator so commissioning can fix the YAML.

Override the VAMP planner's world cuboids at build time when they live outside the cell YAML:

with build_calibration_app(Path("cell.yaml"), cuboids_path=Path("custom.json")) as cal:
    ...

After a camera swap, before the new MxId has calibio intrinsics in the store, opt into the live-defaults fallback so calibration verbs still run:

with build_calibration_app(Path("cell.yaml"), strict_intrinsics=False) as cal:
    cal.run_hand_eye_calibration(...)        # uses camera defaults; logs warning

Transport (direct vs gRPC)

build_calibration_app honors griip_api.mode in the cell YAML:

• mode: direct (default) — wires the in-process PlanningApi and PerceptionApi from mmai_griip_api. Pulls the dev-only mmai_griip_api package as a transitive runtime dep at the call site.
• mode: grpc — dials the gRPC server at griip_api.url (defaults to localhost:50051) using the SDK's own generated stubs. No mmai_griip_api dep needed; the published SDK wheel is enough.

The gRPC channel is closed for you on context-manager exit.

BYO stubs (advanced)

CalibrationApp.__init__ is keyword-only and accepts the planning and perception stubs as injected dependencies. When the cell-app needs a non-standard transport (custom interceptors, an alternate server, test fixtures), construct it directly and skip the factory:

import grpc
from griip_sdk import CalibrationApp
from griip_sdk.calibration.config_store import CalibrationConfigStore
from griip_core.generated import griip_pb2_grpc
from griip_sdk.hardware.mmai_vision_backend import MmaiVisionServiceBackend

channel = grpc.insecure_channel("custom-host:50051")

# vision_backend is the same vention.vision.v1 ABC used everywhere else in the
# SDK (CameraManager, teleop, ...); MmaiVisionServiceBackend adapts an already
# started MmaiVisionClient to it. See "Vision ABC" above for the interface.
vision_backend = MmaiVisionServiceBackend(camera_client, left_source_id="left", right_source_id="right")

cal = CalibrationApp(
    cell=cell,
    config_path=cell_yaml_path,
    robot_client=robot_client,
    vision_backend=vision_backend,
    cam_name="luxonis",
    camera_mxid="...",
    sensor="left",
    store=CalibrationConfigStore(),
    calib_data_root="/data/vention/calib_data",
    planning_stub=griip_pb2_grpc.PlanningServiceStub(channel),
    perception_stub=griip_pb2_grpc.PerceptionServiceStub(channel),
)

# ...drive the verbs as normal, then close `channel` and unregister the
# robot when done (the BYO path skips the factory's cleanup).

Verbs on CalibrationApp

Verb	Effect
cal.run_hand_eye_calibration(on_progress=None, is_cancelled=None, routine=None, ...)	Plan to cell.positions.calibration_joints, run hand-eye, then return the robot to that start pose. routine defaults to griip_sdk.calibration.routines.DEFAULT_ENCIRCLE_ROUTINE. Persists X_cam_flg__{cam}__{mxid}__{sensor}. Returns the camera-to-flange SE3. Raises ConfigError when the YAML field is missing.
cal.last_hand_eye_calibration_timestamp()	datetime (UTC) of the latest hand-eye, or None.
cal.start_environment_session(save_dir=None)	Open an operator-driven scan session. Creates the sequence dir, persists post-crop intrinsics, switches robot to freedrive. Returns the sequence dir.
cal.take_environment_image()	Capture one frame in the active session (stereo, depth, pose). Robot stays in freedrive. Returns the frame index.
cal.end_environment_session()	Restore the robot to normal operation, persist env_scan__{cam}__{mxid} pointing at the sequence dir. Returns the sequence dir.
cal.last_environment_scan_timestamp()	datetime (UTC) of the latest completed scan session, or None.

Release notes

The PyPI release history lists every published version. Each page freezes the README and CHANGELOG as of that release.