griip-sdk

Hardware control and orchestration for bin picking, by Vention.

The SDK owns the robot, gripper, perception, and planning. You own the loop: when to pick, where the part goes, and what to do on failure.

Install

pip install griip-sdk

Python 3.10. Needs a running griip-server. From outside the Vention monorepo, use gRPC mode.

Wheels publish to public PyPI from master via the monorepo nx release pipeline. Pull-request CI only dry-runs. Cells that still keep private extras under /opt/vention/wheels/ can add --find-links /opt/vention/wheels/.

Testing a pull request

Every pull request that touches the SDK publishes a dev build to its own PyPI project, so you can try a branch before it merges:

pip uninstall -y griip-sdk          # both own the `griip_sdk` import package
pip install griip-sdk-dev==<version>

The version is on the pull request's CI summary, as <next release>.dev<run number>. pip show griip-sdk-dev names the commit it was built from. Dev builds depend on released siblings, so a branch that also changes griip-core is only half-covered by one. Never install one on a production cell.

Quickstart

from griip_sdk import build_pick_only_manager

DROP_JOINTS = [-1.5, -1.2, 1.0, -1.4, -1.5, 0.0]   # your drop pose

with build_pick_only_manager("./pick.yaml", part_id="pcb") as manager:
    pick = manager.pick(max_attempts=3)
    if pick.succeeded:
        manager.start_pick_generation()                 # perception runs while you move
        manager.collision_free_move_to_joints(DROP_JOINTS, object_in_tcp=pick.obj_in_tcp)
        manager.release()

Two entry points, same setup under the hood:

• build_pick_only_manager for single-cycle pick-only work (above).
• build_bin_picking_manager for a continuous bin-pick + placement loop; you pass a handle_placement callback and call manager.run().

Both return a context manager that wires up hardware and tears it down on exit.

Handling failures

manager.pick() always returns a PickResult. On failure, failure_reasons holds one reason per attempt in order. failure_reasons[-1] is the last attempt; more than one distinct reason means the cell is failing for shifting reasons.

PickFailureReason	Meaning	Typical response
NO_PARTS_DETECTED	Perception saw no objects.	Request refill.
NO_GRASPABLE_PARTS	Parts visible but none reachable.	Shake / reorient.
GRIP_CHECK_FAILED	Scooped but nothing in hand.	Retry next loop.
MOTION_FAILED	Collision or scoop plan/exec failure.	Alert operator.
PERCEPTION_FAILED	Picking stream died mid-call (gRPC error).	Backoff + retry, or page ops.
PLACEMENT_FAILED	Pick OK, place leg failed (placement mode).	Cell-specific.

pick.attempts_made and pick.detail are for logs and metrics, not control flow. Hardware faults (robot or gripper driver) don't map to a reason: they escape as exceptions, which is your "stop the cell" signal.

A loop that acts on each reason:

from griip_sdk import build_pick_only_manager, PickFailureReason

with build_pick_only_manager("./pick.yaml", part_id="pcb") as manager:
    while True:
        pick = manager.pick(max_attempts=3)

        if pick.succeeded:
            manager.start_pick_generation()
            manager.collision_free_move_to_joints(DROP_JOINTS, object_in_tcp=pick.obj_in_tcp)
            manager.release()
            continue

        if len(set(pick.failure_reasons)) > 1:
            alert_operator(f"cell unstable: {pick.failure_reasons}")
            break

        match pick.failure_reasons[-1]:
            case PickFailureReason.NO_PARTS_DETECTED:
                request_refill()
            case PickFailureReason.NO_GRASPABLE_PARTS:
                shake_bin()
            case PickFailureReason.GRIP_CHECK_FAILED:
                pass                       # SDK already retried; try again next loop
            case PickFailureReason.MOTION_FAILED | PickFailureReason.PERCEPTION_FAILED:
                alert_operator(pick.detail)
                break

Manager methods

Method	Effect
pick(max_attempts=3)	Pick only. Returns a PickResult; robot stays at the prep pose on success.
pick_and_place_once(max_pick_attempts=3, target_pose=...)	Pick, then place when target_pose is given.
start_pick_generation()	Move to the capture pose and run perception. The camera grab is synchronous; perception runs async.
collision_free_move_to_joints(target, object_in_tcp=None, ...)	Collision-aware joint move. Pass object_in_tcp=pick.obj_in_tcp while carrying a part.
release()	Open the gripper and clear object_in_hand.
set_capture_joints(joints)	Change where perception captures from, next capture on.
set_part(part_id)	Swap the active part mid-session. Re-sends Initialize over the existing channel; no hardware or planner re-init.

pick.yaml

The cell YAML points the runner at the robot, gripper, camera, perception backends, and the cell's collision geometry. It does not hold drop poses or loop policy. That is your application code.

Bring your own gripper

Both builders take an optional gripper=. When set, the SDK uses your client instead of building one from the YAML.

Use the SDK's Robotiq 2F-140 directly, so a bench consumer keeps its hardware constants out of the cell YAML:

from griip_sdk import build_pick_only_manager
from griip_sdk.hardware.robotiq_2f140_gripper import Robotiq2F140Gripper

gripper = Robotiq2F140Gripper(serial_port="/tmp/ttyUR", baudrate=115200, modbus_slave_id=9, grip_force=20, release_width=800)

with build_pick_only_manager("./pick.yaml", part_id="pcb", gripper=gripper) as manager:
    ...

Or subclass BaseGripper for a gripper the SDK doesn't ship (a Modbus / EtherNet-IP unit, or an in-memory fake for tests):

from griip_sdk import BaseGripper, GripperMoveResult, build_pick_only_manager

class MyEthernetIPGripper(BaseGripper):
    def get_width(self) -> int: ...
    def get_current_draw(self) -> int: ...
    def is_in_safety_stop(self) -> bool: ...
    def reset_safety(self) -> None: ...
    def reconnect(self) -> None: ...
    def stop(self) -> None: ...
    def close_gripper(self, force_value=400, block=False, timeout=10.0) -> GripperMoveResult: ...
    def open_gripper(self, force_value=400, block=False) -> GripperMoveResult: ...
    def move_gripper(self, width_value, force_value=400, block=False, timeout=None) -> GripperMoveResult: ...

with build_pick_only_manager("./pick.yaml", part_id="pcb", gripper=MyEthernetIPGripper(...)) as manager:
    ...

Two things to know:

• The YAML tool.gripper block still drives manager state (closed_gripper_width, expected_grip_width_range, grip_check_width_margin, min_grip_width_for_current_part). Only the hardware-construction fields (serial_port, baudrate, modbus_slave_id, force, release_width) are ignored when you inject a gripper. The SDK logs this once at INFO.
• The SDK does no gripper-side teardown. If your gripper holds a connection (TCP or serial), close it yourself.

Bring your own camera (vision ABC)

Both builders take an optional vision_backend=. The SDK camera path talks to the vention.vision.v1 Image/Source service ABCs, so the same CameraManager drives the legacy Luxonis camera and the native gRPC cameras (vvis, Leopard) behind one interface. When vision_backend is omitted (or in mock mode) the SDK builds whichever backend the cell YAML camera.backend field selects.

The easiest route is config-only — pick the backend in the cell YAML:

camera:
  backend: "grpc"              # "mmai" (default, legacy Luxonis HTTP) or "grpc"
  target: "127.0.0.1:41051"    # vention.vision gRPC service
  sensor: "full_frame_left"    # primary source id reported by ListSources
  right_source_id: "full_frame_right"
  source_metadata: # only keys the service omits; service values always win
    camera_vendor: "leopard"
    mx_id: "LI-123456"

or build it in code with the same dispatch:

from griip_sdk import build_bin_picking_manager
from griip_sdk.hardware.grpc_vision_backend import build_vision_backend

vision_backend = build_vision_backend("./cell.yaml")  # Mmai or gRPC per camera.backend
with build_bin_picking_manager("./cell.yaml", part_id="pcb", handle_placement=..., vision_backend=vision_backend) as manager:
    ...

GrpcVisionServiceBackend composes the wheel's create_image_service_backend / create_source_service_backend gRPC backends so one object satisfies both ABC seams; build_mmai_vision_backend still builds the Luxonis adapter explicitly. Intrinsics, resolution, and stereo baseline are resolved from the backend's list_sources, so each camera reports its own calibration through the same seam — native services own their calibration (no calibio upload), and source_metadata must carry baseline / camera_vendor / mx_id (from the service or the YAML overlay) or calibration resolution fails loudly.

The camera layer imports the vention.vision.v1 protos and vision_service_abc ABCs, so it needs vention-firmware-grpc-client>=1.3.0.

Swapping parts mid-session

with build_pick_only_manager("./pick.yaml", part_id="pcb_rev_a") as manager:
    for _ in range(5):
        manager.pick(max_attempts=3)

    manager.set_part("pcb_rev_b")     # ValueError if part_id isn't in pick.yaml

    for _ in range(5):
        manager.pick(max_attempts=3)

Only works if both parts use the same gripper. If they don't, build a fresh manager for each.

Manual lifecycle (without with)

The with form is crash-safe and recommended. For a long-running daemon that needs a global-style handle, drive the lifecycle yourself:

_ctx = build_pick_only_manager("./pick.yaml", part_id="pcb")
manager = _ctx.__enter__()
try:
    manager.pick(max_attempts=3)
    # manager is usable anywhere in the process
finally:
    _ctx.__exit__(None, None, None)   # required: closes gRPC + hardware

This is an escape hatch. Skip __exit__ and you leak the gRPC channel and hardware connections.

Grasp targets, your motion

For a cell where the application moves the robot itself. The SDK takes and stores pictures, runs perception on a stored picture when asked, and keeps one queue of ranked grasps. It never commands motion.

from griip_sdk import build_grasp_provider

with build_grasp_provider("pick.yaml", part_id="smallest_splice", robot=MyRmiRobot(rmi)) as sdk:
    # First round: one picture of each pallet, then start on the left.
    move_robot_to(LEFT_CAPTURE_POSE)     # your motion
    sdk.capture("left")                  # take a picture now, store it as "left"
    move_robot_to(RIGHT_CAPTURE_POSE)
    sdk.capture("right")                 # take a picture now, store it as "right"
    sdk.process("left")                  # start perception on the "left" picture, in the background

    while True:
        # Left pallet
        target = sdk.next_target()       # wait for the "left" run, pop its best reachable grasp; None = nothing graspable
        sdk.process("right")             # start perception on the "right" picture; it runs during the steps below
        picked_left = target is not None
        if picked_left:
            move_to(target.approach_pose_mm_deg)         # your motion to the approach ([x, y, z, w, p, r], mm/deg)
            move_linear(target.grasp_pose_mm_deg)        # straight in along the tool Z
            close_gripper()
            move_linear(target.approach_pose_mm_deg)     # straight back out
            sdk.capture("left")          # new picture of the left pallet for the next round, taken now
            move_robot_to(PLACE_POSE)
            open_gripper()

        # Right pallet
        target = sdk.next_target()       # wait for the "right" run, pop its best reachable grasp
        sdk.process("left")              # start perception on the new "left" picture; runs while we pick right
        picked_right = target is not None
        if picked_right:
            move_to(target.approach_pose_mm_deg)
            move_linear(target.grasp_pose_mm_deg)
            close_gripper()
            move_linear(target.approach_pose_mm_deg)
            sdk.capture("right")         # new picture of the right pallet for the next round
            move_robot_to(PLACE_POSE)
            open_gripper()

        if not picked_left and not picked_right:
            break                        # both pallets came back empty

Five verbs. capture(tag) stores a picture with the robot pose and joints at that moment, so park the robot first; the tag is any string you choose, one stored picture per tag. captures() lists the stored pictures, delete(tag) removes one. process(tag) runs perception on a stored picture in the background, one image at a time. next_target(timeout=None) waits for in-flight runs and pops the best grasp; it returns None when the last picture had nothing graspable, when the run failed (captures() carries the state and the reason), or on timeout.

The results of a run replace the queue, so pop before starting the next run. Re-capturing a tag replaces its picture, and a run still going for the old picture is dropped.

Each GraspTarget answers what a motion-owning app needs to know:

• capture_tag says which picture, and therefore which pallet, it came from; object_index which part in it. Several grasps can come back per part.
• approach_pose is where to go first; grasp_pose is where the gripper closes, approach_offset (from the grasp definition) further along the tool Z. Both are TCP poses in the robot base frame. approach_pose_mm_deg / grasp_pose_mm_deg give them as [x, y, z, w, p, r]: millimetres, and degrees about X, Y then Z, the pendant convention. The SE3 forms stay in metres for code that composes transforms.
• at_flange(pose) converts either one for a controller whose TCP is the flange (tool.tcp_offset is returned as tcp_offset).
• plunge_pose / scoop_pose are the SDK's own guarded-scoop phases. The scoop can end past the grasp point, so never move to it without a force stop.
• approach_joint_angles / grasp_joint_angles are your robot's IK solutions when it checked the grasp (see below).
• approach_trajectory (only with planning.plan_approach_paths: true, see below; otherwise None) is a collision-free joint path to the approach: VAMP checks the arm plus the gripper and camera spheres against the cell cuboids. Run it as joint moves with no blending (FINE): only the straight joint-space segments between waypoints are checked, and blending cuts corners nobody checked. sdk.plan(start_joints, goal_joints) gives the same for any other move (to a capture pose, to place). The straight move from the approach to grasp_pose and back is treated as collision-free: the approach offset in the grasp definition is what keeps it clear.
• Everything the server used for this grasp, so an app running its own plunge/scoop never re-reads grasps.json: plunge_velocity, scoop_velocity, force_torque_sensor_threshold, retract_move_distance, and the gripper parameters (executor_type is "parallel_jaw", "suction" or "three_finger"; executor_params is that message, e.g. prep_width, engaged_width, grip_check_min_width, grip_check_max_width). Units are those of grasps.json.
• object_pose_in_world (and object_pose_mm_deg) is the part as perception saw it, tilt included; object_in_tcp is the part relative to the TCP at the end of the scoop.

Same pick.yaml as a self-driving cell: part, grasps, tool.tcp_offset, camera, hand-eye. No home_bin* poses are needed; the capture pose is whatever the robot reports. No gripper is built or actuated.

Only if the pictures show a container with walls: capture(tag, bin_number=N) attaches the bin_N_* cuboids from cuboids.json to the request so grasps through a wall or below the floor are rejected. N must be listed in runtime.bins.available. It is not a taught pose. Open pallets need none of this.

Your own robot (no VRMCS)

When the app owns the robot, say a Fanuc over RMI, set it up in pick.yaml and pass any object with these three methods, in pendant units. The SDK opens no connection of its own.

robot:
  driver: external
  model: crx25ia            # a robot in griip-server's vamp-planner (planning and calibration)
  joint_convention: fanuc   # joints in and out as the pendant shows them
planning:
  plan_approach_paths: false  # true: each target also gets a collision-free approach_trajectory

By default you get targets your IK can reach and move there your own way. Turn on plan_approach_paths and each target also carries a collision-free path to its approach, planned by VAMP in griip-server; nothing is installed on your side either way.

class MyRmiRobot:
    def __init__(self, rmi):
        self._rmi = rmi              # your RMI session; the SDK never opens one

    def get_tcp_pose(self) -> list[float]:
        # Current TCP in UFrame 0, active UTool, [x, y, z, w, p, r] in mm and degrees.
        return self._rmi.read_cartesian_position()

    def get_joint_angles(self) -> list[float]:
        # In robot.joint_convention: pendant degrees with `fanuc`.
        return self._rmi.read_joint_angles()

    def inverse_kinematics(self, tcp_pose: list[float], seed_joint_angles: list[float]) -> list[float] | None:
        # None only when tcp_pose is unreachable; raise for anything else (not ready, comms).
        return self._rmi.inverse_kinematics(tcp_pose, seed_joint_angles)


with build_grasp_provider("pick.yaml", part_id="smallest_splice", robot=MyRmiRobot(rmi)) as sdk:
    assert rmi.active_utool_offset_mm() == sdk.tcp_offset_mm   # optional: check the controller tool once
    ...

• Threads. The SDK calls these methods only on the thread that called capture (get_tcp_pose, get_joint_angles) or next_target, never from a background thread, so they serialize with the app's motion if the app calls them between moves. Call them while the arm moves and they need the app's robot lock.
• Calls per next_target. One get_joint_angles, then two inverse_kinematics for the grasp it returns (approach, then grasp), plus one path plan on the server when planning is on (no robot call). Each skipped candidate adds one or two IK calls and at most one plan.
• Seeds. The approach is seeded with the current joints read at the start of that next_target; the grasp with the approach solution, the configuration the arm is in when it starts the straight move. The path is planned from those same current joints.
• Joints. Every joint list in and out (your methods, the IK solutions, the trajectory, sdk.plan) is in robot.joint_convention. With fanuc that is pendant degrees and the SDK does the conversion VAMP needs (radians, and the pendant's J3, measured from the horizontal, re-expressed relative to link 2). The default model is URDF radians.
• Errors. None drops that candidate as unreachable. An exception from your robot propagates out of next_target and the candidate stays at the front of the queue for the next call.
• Frames. Poses are TCP poses in the robot base frame (UFrame 0) at tool.tcp_offset, which sdk.tcp_offset_mm returns; the active UTool must match it, or use UTool 0 with target.at_flange(...).

With an external robot nothing talks to VRMCS. The server has no IK: it sends every grasp that clears the bin walls, and next_target drops what your IK can't reach (and, with planning on, what VAMP can't find a collision-free path to), attaching both IK solutions and, with planning on, the path.

Calibration

The SDK exposes the two calibration primitives — hand-eye and operator-driven environment scanning. The cell YAML is the source of truth for the calibration starting pose (positions.calibration_joints); commissioning authors it, the SDK reads it, and the operator app never edits it at runtime.

The simplest hand-eye run — build the app, call the verb:

from griip_sdk import build_calibration_app

with build_calibration_app("cell.yaml") as cal:
    x_cam_flg = cal.run_hand_eye_calibration()

The SDK plans to cell.positions.calibration_joints, runs the bundled sphere routine, returns the robot to that start pose, and persists the result. The fuller form below adds progress reporting and the operator-driven scan session:

On a cell with robot.driver: external, pass the app's CalibrationRobot: a GraspRobot plus execute_joint_path(waypoints). griip-server's VAMP plans every move for robot.model (calibration always plans, whatever plan_approach_paths says), your IK solves the pose targets, and your robot executes the path, in robot.joint_convention. Environment scanning needs freedrive and is not available on an external robot.

with build_calibration_app("cell.yaml", robot=MyRmiCalibrationRobot(rmi)) as cal:
    x_cam_flg = cal.run_hand_eye_calibration()

from pathlib import Path

from griip_sdk import build_calibration_app

with build_calibration_app(Path("cell.yaml")) as cal:
    # Hand-eye: the SDK plans to cell.positions.calibration_joints, runs the
    # bundled sphere routine, then returns the robot to the start pose.
    # Override `routine=` only if your cell geometry truly needs a non-default recipe.
    x_cam_flg = cal.run_hand_eye_calibration(
        on_progress=lambda pct, status, msg: print(pct, status, msg),
        is_cancelled=lambda: False,
    )
    cal.last_hand_eye_calibration_timestamp()       # datetime | None

    # Environment scanning: operator-driven session. start_environment_session
    # switches the robot into freedrive; take_environment_image grabs one
    # frame at the current pose; end_environment_session restores normal
    # operation and persists the entry.
    cal.start_environment_session()
    for _ in range(num_frames_the_app_wants):
        # operator manually repositions the robot between captures
        cal.take_environment_image()
    sequence_dir = cal.end_environment_session()
    cal.last_environment_scan_timestamp()           # datetime | None

If cell.positions.calibration_joints is unset, run_hand_eye_calibration raises griip_sdk.ConfigError with a message naming the missing field and the config file. Surface this to the operator so commissioning can fix the YAML.

Override the VAMP planner's world cuboids at build time when they live outside the cell YAML:

with build_calibration_app(Path("cell.yaml"), cuboids_path=Path("custom.json")) as cal:
    ...

After a camera swap, before the new MxId has calibio intrinsics in the store, opt into the live-defaults fallback so calibration verbs still run:

with build_calibration_app(Path("cell.yaml"), strict_intrinsics=False) as cal:
    cal.run_hand_eye_calibration(...)        # uses camera defaults; logs warning

Transport (direct vs gRPC)

build_calibration_app honors griip_api.mode in the cell YAML:

• mode: direct (default) — wires the in-process PlanningApi and PerceptionApi from griip_server. Pulls the dev-only griip_server package as a transitive runtime dep at the call site.
• mode: grpc — dials the gRPC server at griip_api.url (defaults to localhost:50051) using the SDK's own generated stubs. No griip_server dep needed; the published SDK wheel is enough.

The gRPC channel is closed for you on context-manager exit.

BYO stubs (advanced)

CalibrationApp.__init__ is keyword-only and accepts the planning and perception stubs as injected dependencies. When the cell-app needs a non-standard transport (custom interceptors, an alternate server, test fixtures), construct it directly and skip the factory:

import grpc
from griip_sdk import CalibrationApp
from griip_sdk.calibration.config_store import CalibrationConfigStore
from griip_core.generated import griip_pb2_grpc
from griip_sdk.hardware.mmai_vision_backend import MmaiVisionServiceBackend

channel = grpc.insecure_channel("custom-host:50051")

# vision_backend is the same vention.vision.v1 ABC used everywhere else in the
# SDK (CameraManager, teleop, ...); MmaiVisionServiceBackend adapts an already
# started MmaiVisionClient to it. See "Vision ABC" above for the interface.
vision_backend = MmaiVisionServiceBackend(camera_client, left_source_id="left", right_source_id="right")

cal = CalibrationApp(
    cell=cell,
    config_path=cell_yaml_path,
    robot_client=robot_client,
    vision_backend=vision_backend,
    cam_name="luxonis",
    camera_mxid="...",
    sensor="left",
    store=CalibrationConfigStore(),
    calib_data_root="/data/vention/calib_data",
    planning_stub=griip_pb2_grpc.PlanningServiceStub(channel),
    perception_stub=griip_pb2_grpc.PerceptionServiceStub(channel),
)

# ...drive the verbs as normal, then close `channel` and unregister the
# robot when done (the BYO path skips the factory's cleanup).

Verbs on CalibrationApp

Verb	Effect
cal.run_hand_eye_calibration(on_progress=None, is_cancelled=None, routine=None, ...)	Plan to cell.positions.calibration_joints, run hand-eye, then return the robot to that start pose. routine defaults to griip_sdk.calibration.routines.DEFAULT_ENCIRCLE_ROUTINE. Persists X_cam_flg__{cam}__{mxid}__{sensor}. Returns the camera-to-flange SE3. Raises ConfigError when the YAML field is missing.
cal.last_hand_eye_calibration_timestamp()	datetime (UTC) of the latest hand-eye, or None.
cal.start_environment_session(save_dir=None)	Open an operator-driven scan session. Creates the sequence dir, persists post-crop intrinsics, switches robot to freedrive. Returns the sequence dir.
cal.take_environment_image()	Capture one frame in the active session (stereo, depth, pose). Robot stays in freedrive. Returns the frame index.
cal.end_environment_session()	Restore the robot to normal operation, persist env_scan__{cam}__{mxid} pointing at the sequence dir. Returns the sequence dir.
cal.last_environment_scan_timestamp()	datetime (UTC) of the latest completed scan session, or None.

Release notes

The PyPI release history lists every published version. Each page freezes the README and CHANGELOG as of that release.