chakra-marina 0.2.0

A framework for evaluating and optimizing agents and models using sandboxed environments.

Marina

Marina is a fork of Harbor, a framework for evaluating and optimizing AI agents and language models. You can use Marina to:

• Evaluate arbitrary agents like Claude Code, OpenHands, Codex CLI, and more.
• Build and share your own benchmarks and environments.
• Conduct experiments in thousands of environments in parallel through providers like Daytona and Modal.
• Generate rollouts for RL optimization.

marina run -p examples/tasks/hello-screenshot --agent openhands-sdk --model openrouter/openai/gpt-5.5 --ae LLM_SUPPORTS_VISION=true

Changes

We track upstream Harbor closely, with a few additions of our own.

Agent kwargs (--ak key=value)

• disable_builtin_tools — drop the default terminal/file-editor/task-tracker tools
• disable_stuck_detection — turn off the SDK's stuck-agent detection

Environment variables (--ae NAME=value)

• LLM_SUPPORTS_VISION — force vision support for models LiteLLM misclassifies
• OPENROUTER_REASONING_ENABLED / _EXCLUDE — control OpenRouter reasoning (auto-on for Opus 4.7)
• OPENROUTER_VERBOSITY — set OpenRouter verbosity
• LLM_THINKING_DISPLAY — thinking display mode (default summarized)
• SYSTEM_MESSAGE_SUFFIX / USER_MESSAGE_SUFFIX — append text to the OpenHands SDK system / user prompts
• AWS_BEARER_TOKEN_BEDROCK — bearer-token auth for Bedrock

Vision support also writes MCP image observations to /logs/agent/trajectory-images/, referenced from the ATIF trajectory.

Installation

Marina is published to PyPI as chakra-marina.

Install the CLI. To put marina on your PATH:

uv tool install chakra-marina

Or with pip:

pip install chakra-marina

Add extras for cloud providers — e.g. chakra-marina[daytona], or chakra-marina[cloud] for all providers and chakra-marina[all] for everything.

Example: Running a task

Run the bundled hello-screenshot task locally with Docker. It's a vision smoke test: the agent calls an MCP take_screenshot tool, sees a solid-color image, and reports the color

export LLM_API_KEY=<YOUR-KEY>
marina run -p examples/tasks/hello-screenshot \
   --agent openhands-sdk \
   --model openrouter/openai/gpt-5.5 \
   --ae LLM_SUPPORTS_VISION=true \
   --ae SYSTEM_MESSAGE_SUFFIX="You run fully autonomously — never ask for confirmation."

--ae LLM_SUPPORTS_VISION=true enables vision (required for this task), and --ae SYSTEM_MESSAGE_SUFFIX="..." appends text to the OpenHands SDK system prompt. Both apply only to the openhands-sdk agent (see Changes).

To run on a cloud provider (like Daytona) instead of local Docker, pass the --env flag:

export LLM_API_KEY=<YOUR-KEY>
export DAYTONA_API_KEY=<YOUR-KEY>
marina run -p examples/tasks/hello-screenshot \
   --agent openhands-sdk \
   --model openrouter/openai/gpt-5.5 \
   --ae LLM_SUPPORTS_VISION=true \
   --env daytona

When running a whole benchmark (many tasks), raise --n-concurrent to fan out across hundreds or thousands of environments in parallel.

To see all supported agents, and other options run:

marina run --help