Skip to main content

DeepAgents harness profile for GigaChat

Project description

deepagents-gigachat

harness-bench-fast: +13.4 pp with the GigaChat profile

A HarnessProfile for deepagents tuned for GigaChat models. On the harness-bench-fast 313-task agent benchmark (v0.9.0) the profile reaches 269 / 313 (85.9 %) with GigaChat-3-Ultra — see the Benchmark section for methodology and historical results.

The profile replaces the default deepagents system prompt, rewrites the descriptions of file and shell tools (ls, read_file, write_file, glob, grep, edit_file, execute) to match GigaChat's tool-calling behavior, and adds middleware for structured reasoning, shell safety, path normalization, memory-task nudges, and loop breaking.

Once installed, the profile is registered automatically via the deepagents.harness_profiles entry point — no code changes required.

Structure

  • deepagents_gigachat/harness_profile.py — GigaChat HarnessProfile, middleware, tool overrides
  • deepagents_gigachat/prompts.py — system prompt variants (native_fs, tool_agnostic, …)
  • deepagents_gigachat/__init__.py — exports register_harness(), set_workspace_path(), …

Requirements

  • Python 3.12+
  • deepagents >= 0.6.7 (0.6.x filesystem API; see Filesystem backend)
  • uv (for dependency installation and execution)

Installation

uv sync

Downstream users can install the published package from PyPI with:

pip install deepagents-gigachat

Configuration

Provide one of the authentication options in your shell environment. If your launcher loads dotenv files, for example deepagents-code, these values can also live in .env:

  • GIGACHAT_CREDENTIALS
  • or GIGACHAT_USER + GIGACHAT_PASSWORD

Optional GigaChat settings:

GIGACHAT_BASE_URL="https://gigachat.sberdevices.ru/v1"
GIGACHAT_MODEL="GigaChat-3-Ultra"
GIGACHAT_VERIFY_SSL_CERTS=False
GIGACHAT_PROFANITY_CHECK=False

Use With deepagents

Install this package into the same Python environment where deepagents runs:

pip install deepagents-gigachat

For local development, install the built wheel instead:

uv build
uv pip install dist/*.whl

After installation, deepagents discovers the profile automatically through the deepagents.harness_profiles entry point:

[project.entry-points."deepagents.harness_profiles"]
gigachat = "deepagents_gigachat:register_harness"

The package entry point is named gigachat for discovery. The harness profile is registered under both provider keys: gigachat for model specs such as gigachat:GigaChat-3-Ultra, and giga as a compatibility alias.

For a minimal inline create_deep_agent + GigaChat snippet, see Examples. It works as long as your GigaChat credentials are available in environment variables (GIGACHAT_CREDENTIALS or GIGACHAT_USER + GIGACHAT_PASSWORD).

Filesystem backend

This profile is tuned for runners that use a local filesystem backend with virtual_mode=True — the setup used by harness-bench-fast and recommended for deepagents-code when agents work inside an isolated workspace directory.

With virtual_mode=True:

  • File tools (read_file, write_file, edit_file, grep, glob, ls) treat the workspace root as /. Use relative paths in tool calls: notes.md, src/utils.py — not host absolute paths like /Users/you/project/notes.md.
  • execute still runs in the host shell working directory (the workspace folder). Shell commands must also use relative paths: cat data.csv works, cat /data.csv does not.

The profile includes PathNormalizerMiddleware to strip leading / from grep/glob results so the agent writes relative paths into output files.

Always pass virtual_mode explicitly when constructing the backend — in deepagents 0.6.x the default is still False, but the 0.6 API expects an explicit choice:

from deepagents import create_deep_agent
from deepagents.backends import LocalShellBackend
from langchain_gigachat import GigaChat

backend = LocalShellBackend(root_dir=".", virtual_mode=True)
agent = create_deep_agent(
    model=GigaChat(model="GigaChat-3-Ultra"),
    backend=backend,
)

If you run with virtual_mode=False, filesystem tool paths follow the real host filesystem instead; the relative-path prompts in this profile will not match that mode.

Use With deepagents-code

Step-by-step setup for using GigaChat as the default model in deepagents-code through its config file.

1. Install deepagents-code, the GigaChat provider, and this plugin

All three must end up in the same Python environment so that deepagents-code can both construct a GigaChat model and discover the harness profile via the deepagents.harness_profiles entry point:

uv tool install deepagents-code --with langchain-gigachat,deepagents-gigachat

(or pip install ... if you're not using uv).

2. Provide credentials

GigaChat accepts two authentication styles. Pick one.

Option A: Authorization Key (one base64-encoded string). Get the key from developers.sber.ru → your project → credentials section, then export it:

export GIGACHAT_CREDENTIALS="<base64-encoded auth key>"

Option B: User + password. If you have a user/password pair instead of a single key:

export GIGACHAT_USER="<your client id>"
export GIGACHAT_PASSWORD="<your client secret>"

You can also put either pair into a .env file next to where you launch deepagents-code — it reads .env on startup. The plugin itself never parses these variables: langchain-gigachat picks them up when it constructs the model.

3. Configure ~/.deepagents/config.toml

Create the file (the directory may not exist yet — mkdir -p ~/.deepagents first) and put the snippet below into it. Each block is annotated.

[models]
# The model used when you launch `deepagents-code` with no extra flags.
# Format: "<provider>:<model name>". The provider key here ("gigachat")
# is the same one this plugin registers its harness profile under.
default = "gigachat:GigaChat-3-Ultra"

[models.providers.gigachat]
# Models exposed to the "/model" picker. Add or remove freely.
models = [
    "GigaChat-3-Ultra",
    "GigaChat-2-Max",
    "GigaChat-Max",
    "GigaChat-Pro",
    "GigaChat",
]
# Tells `deepagents-code` which Python class to instantiate when a
# `gigachat:*` spec is requested.
class_path = "langchain_gigachat.chat_models.gigachat:GigaChat"
# If you authenticate via GIGACHAT_CREDENTIALS, this line wires it up.
# Remove this line if you use GIGACHAT_USER + GIGACHAT_PASSWORD instead.
api_key_env = "GIGACHAT_CREDENTIALS"

[models.providers.gigachat.params]
# Constructor kwargs passed straight to `GigaChat(...)`. Anything that
# `langchain_gigachat.GigaChat` accepts can go here.
base_url = "https://gigachat.sberdevices.ru/v1"
verify_ssl_certs = false
profanity_check = false
timeout = 600
# Optional sampling knobs (defaults are sensible; uncomment to override):
# temperature = 0.0
# top_p = 1.0
# repetition_penalty = 1.0

[models.providers.gigachat.profile]
# Tells the profile resolver that this provider supports tool
# calling and which model to default to when the user types just
# "gigachat" without a model name.
tool_calling = true
default_model_hint = "GigaChat-3-Ultra"

4. Run deepagents-code

deepagents-code

On startup, deepagents-code loads the config, instantiates GigaChat with the parameters above, and deepagents automatically picks up this plugin's harness profile via its deepagents.harness_profiles entry point — so GigaChat-specific system prompt, tool description overrides and the think middleware are applied without any extra code.

Switching models

Three independent ways to override the default at runtime:

  • Inside deepagents-code: type /model gigachat:GigaChat-Pro to switch the current session.
  • From the shell, per-launch: deepagents-code --model gigachat:GigaChat-Max.
  • From the environment: set GIGACHAT_MODEL=GigaChat-Pro before launching. (This is honoured by langchain-gigachat itself when the model name isn't pinned in the config.)

Self-hosted / IFT GigaChat endpoint

Point base_url at your custom host. For Sber's internal IFT, for example:

[models.providers.gigachat.params]
base_url = "https://gigachat.ift.sberdevices.ru/v1"

Everything else stays the same.

Examples

Runnable examples live in examples/. The simplest one is examples/basic_agent.py: it constructs a GigaChat model, wraps it in create_deep_agent, and asks a single question. Run it with:

uv run python examples/basic_agent.py

Or run this minimal inline example:

from deepagents import create_deep_agent
from langchain_gigachat import GigaChat

agent = create_deep_agent(
    model=GigaChat(model="GigaChat-3-Ultra"),
    system_prompt="You are a helpful assistant.",
)
result = agent.invoke({"messages": "Hi! What can you do?"})

Benchmark

The benchmark used to validate every profile version of this plugin now lives in its own repo: ai-forever/harness-bench-fast. It is a self-contained agent evaluation (currently 313 tasks, v0.9.0) covering file creation/editing, refactors, project-wide grep/glob, CSV / JSON / JSONL / YAML / TOML / INI / XLSX / SQLite pipelines, pytest-graded implementations, composite multi-step pipelines, merge/conflict resolution, terminal-style parsing, policy/action JSON tasks, and MEMORY.md discipline. Every verifier is mechanical — no LLM-as-judge.

Latest results on the full 313-task set, GigaChat-3-Ultra at gigachat.sberdevices.ru/v1, LocalShellBackend(virtual_mode=True):

Configuration PASS / 313 %
deepagents + this plugin 269 / 313 85.9 %

Historical uplift on the earlier 231-task subset of the same bench:

Configuration PASS / 231 % Δ
stock deepagents, no profile 164 / 231 71.0 %
deepagents + this plugin (v9) 195 / 231 84.4 % +31 (+13.4 pp)

Current middleware stack: ThinkToolMiddleware, ShellSafetyMiddleware, PathNormalizerMiddleware, MemoryTaskMiddleware, LoopBreakerMiddleware, optional ToolContractMiddleware, plus base_system_prompt and tool description overrides.

Lint

Linting, tests, and package build checks are required in CI:

uv run ruff check .
uv run pytest
uv build

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

deepagents_gigachat-0.0.3.tar.gz (22.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

deepagents_gigachat-0.0.3-py3-none-any.whl (23.7 kB view details)

Uploaded Python 3

File details

Details for the file deepagents_gigachat-0.0.3.tar.gz.

File metadata

  • Download URL: deepagents_gigachat-0.0.3.tar.gz
  • Upload date:
  • Size: 22.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.11.15 {"installer":{"name":"uv","version":"0.11.15","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

File hashes

Hashes for deepagents_gigachat-0.0.3.tar.gz
Algorithm Hash digest
SHA256 7864659e2627de2edc014a7f3e55d33cccd96a584f250ea1ed14375366098d51
MD5 bd9312b54d1de6b287789cb661a38348
BLAKE2b-256 8963d90f3a0d0eca8955ae64c87853a460e9d9664a1573318a206e5ae174fcf2

See more details on using hashes here.

File details

Details for the file deepagents_gigachat-0.0.3-py3-none-any.whl.

File metadata

  • Download URL: deepagents_gigachat-0.0.3-py3-none-any.whl
  • Upload date:
  • Size: 23.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.11.15 {"installer":{"name":"uv","version":"0.11.15","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

File hashes

Hashes for deepagents_gigachat-0.0.3-py3-none-any.whl
Algorithm Hash digest
SHA256 131598200c98eebbeb421d3fad1962abebb7f1ed5417052ae93ed4f18964c3aa
MD5 fc20b7d5c72aa58f9ea636646b9a13df
BLAKE2b-256 1d3437cdaf709713b6ece49e4eb421f9a3a61038acb794352267f26e59f8adc6

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page