seamark

The command line for Seamark: run Harbor evaluation tasks in the cloud and follow them from a terminal or a coding agent. seamark run replaces harbor run: tasks are uploaded once, trials are queued and run in Seamark's cloud, and logs, trajectories, results, QA verdicts and costs come back through seamark status, logs, pull, summary and qa.

Install

uv tool install seamark      # or: pipx install seamark

Datasets from the Harbor registry (seamark run -d NAME@VERSION, seamark ls datasets) need Harbor's registry client:

uv tool install 'seamark[harbor]'

Python 3.13 or newer; uv tool install fetches it when needed.

Sign in

Create an API key at https://seamark.cloud/settings/keys, then:

seamark login        # paste the key; it is checked and saved (readable only by you)
seamark whoami       # the organization, scope and server the key acts on

SEAMARK_API_KEY and SEAMARK_API_URL in the environment win over the saved key and the default server. In scripts, pipe the key in (printf %s "$KEY" | seamark login) or set SEAMARK_API_KEY.

Use

seamark checks ./my-task                           # the upload checks, run locally
seamark run ./my-task -a codex -m openai/gpt-6-sol # upload and queue a trial
seamark status                                     # progress of your experiments
seamark logs TRIAL_ID                              # a trial's logs
seamark qa TRIAL_ID                                # did the agent earn its reward?
seamark pull TRIAL_ID                              # its artifacts
seamark skill install                              # teach a coding agent to drive seamark

Every command takes --help, and --json for scripts.