markov-agent

A coding agent following the principles defined in the SKILL.state paper:

Runtime architecture that replaces append-only conversational history with an explicit, mutable execution state. At each execution step, the model receives only the immutable skill specification, the current structured execution state, and the latest observation. Intermediate reasoning is discarded immediately after producing a validated state update, preventing prompt growth with execution history.

Hence, the context given to the agent is modeled similar to a Markov process: the next state depends only on the current one and the (observed result of the) action taken.

Overview

Although the paper specifies tasks like inventory management or CTF, I wanted to use it for a coding agent and test it in tasks I normally throw at Pi and see what happens.

The action space is: bash. Nothing more and nothing less. While most coding agents have some specially guardrailed tools (read, patch, etc.), there is research proving that powerful LMs can do better without:

Predefined tools improve performance for models with weaker bash proficiency, whereas bash-capable models can operate effectively with a bash-only interface and achieve substantially lower cost, especially on command-line-centric tasks.

Installation

uv add markov-agent

Usage

TUI

uv tool install markov-agent
markov  # --model defaults to mistral:mistral-large-4

As a library

skill_state.py is the whole implementation of the paper. Define your state as a pydantic model plus an immutable instruction block, then consume the StepEvents of run(). agent.py is a small coding-agent domain to copy from.

If you only want the core and not the terminal UI (nor its textual dependency), copy skill_state.py and prompt_template.jinja into your project: they import nothing else from this package, and lmdk + jinja2 are their only requirements.

Development

This project uses uv, just and prek.

See justfile for pre-built commands used along pre-commit hooks and CI.

Made with mold template

References

@misc{badhe2026skillstatescalablelonghorizonagent,
      title={SKILL.state: Scalable Long-Horizon Agent Skills},
      author={Sanket Badhe and Priyanka Tiwari and Jonghyun Chung},
      year={2026},
      eprint={2608.26263},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2608.26263},
}