nova-harness 0.1.0

Let a model do the work. Keep your hands on the controls.

Automation for any LiteLLM model, with permissions and a human veto built in.

from NovaHarness.harness import Agent

agent = Agent("gpt-4o-mini", root="./my-project")
print(agent.run("Find every TODO in this project and write them to TODO.md"))

The agent reads and searches freely. When it wants to write TODO.md, it stops and asks you:

Tool 'write_file' requires approval.
  reason: tool in ask list
  path: "TODO.md"
  content: "# TODOs\n\n- src/app.py:41: handle empty input\n..."
  3600s left to answer
Approve? [y/n]: _

That is the whole idea. The model proposes, the harness decides.

Why a harness?

A model alone only writes text. To automate real work it needs to read files, run commands and browse, and every one of those is a way for it to go wrong. The harness is the layer in between:

• It runs the loop. Ask the model, run the tools it requests, feed results back, repeat until it answers.
• It enforces the rules. Every tool call is checked against a policy before it runs.
• It keeps a human in charge. Risky calls wait for a y. No answer means no.
• It stays small. About 1,900 lines, so you can read all of it before you trust it.

Install

pip install nova-harness

Python 3.10 or newer. Give LiteLLM your provider key (export OPENAI_API_KEY=...) or point it at a local model. For DuckDuckGo web search: pip install "nova-harness[search]".

git clone -b nova-harness https://github.com/HmedNejjar/Nova.git
cd Nova
pip install -e ".[dev]"
pytest

What a run looks like

sequenceDiagram
    autonumber
    actor You
    participant R as Runtime
    participant M as Model
    participant P as Policy
    participant H as Approver
    participant X as Executor

    You->>R: agent.run("write the summary to SUMMARY.md")
    R->>M: history + tool specs
    M-->>R: call write_file(path, content)
    R->>P: evaluate(call)
    P-->>R: ask
    R->>H: approve?
    H-->>R: yes
    R->>X: validate args, run with timeout, cap output
    X-->>R: ok
    R->>M: history + result
    M-->>R: "Done, SUMMARY.md written."
    R-->>You: "Done, SUMMARY.md written."

Every outcome comes back to the model as a result, even failures. It sees ok, error (with a message it can act on) or denied, and adapts.

flowchart TD
    call["Tool call"] --> known{"Tool exists?"}
    known -->|no| e1["error: lists the tools that do"]
    known -->|yes| args{"Arguments parsed?"}
    args -->|no| e2["error: resend"]
    args -->|yes| pol{"Policy"}
    pol -->|allow| exec
    pol -->|ask| human{"Human approves?"}
    human -->|yes| exec
    human -->|"no / timeout / error"| den
    pol -->|deny| den["denied: 'Do not retry.'"]
    den --> cnt{"Same call denied<br/>max_denials times?"}
    cnt -->|yes| abort(["PolicyAbortError"])
    cnt -->|no| back["back to the model"]
    exec["Executor: check types, run in a thread with a timeout, cap output"] --> ok["ok"]
    exec -->|"bad args, crash, timeout"| e3["error"]

Safe by default

Reads run on their own. Anything that changes the world asks first.

File tools can only see the root folder you give the agent. web_search appears only when you plug in a search backend.

Make it yours

Add a tool

A typed function with a docstring. The docstring tells the model what it does, the type hints become its schema.

from NovaHarness.harness import Agent
from NovaHarness.tools import tool

@tool
def word_count(text: str) -> int:
    """Count the words in a piece of text."""
    return len(text.split())

agent = Agent("gpt-4o-mini", tools=[word_count], policy={"word_count": "allow"})

Your tool asks for approval until a policy says otherwise, so a new tool is never silently trusted.

Change the rules

from NovaHarness.harness import Agent, Policy

# Adjust the defaults...
Agent("gpt-4o-mini", policy={"run_command": "deny", "write_file": "allow"})

# ...or write the rulebook yourself. Anything unlisted is refused.
Agent("gpt-4o-mini", policy=Policy(
    allow=frozenset({"read_file", "list_directory", "calculator"}),
    default="deny",
))

If a tool appears in more than one list, the strictest wins: deny beats ask beats allow.

Change who answers

The default prompt is the terminal. For a web app, a bot, or a cron job, subclass ToolApprover:

from NovaHarness.harness import Agent, ToolApprover

class HeadlessApprover(ToolApprover):
    def approve(self, tool_call, reason):
        return tool_call.tool_name == "fetch_url"   # fetch pages, nothing else

Agent("gpt-4o-mini", approver=HeadlessApprover())

The base class denies everything, so a forgotten override fails closed.

Change the model

from NovaHarness.harness import Agent

Agent("gpt-4o-mini")
Agent("anthropic/claude-sonnet-5-5", temperature=0.2)
Agent("ollama_chat/llama3.1", api_base="http://localhost:11434")   # fully local

Or pass any object with a generate(messages, tools) method. That is how the tests run without a network.

It remembers

Conversations start empty, but the agent can keep what matters. Facts (name, preferences) and notes (recent context) live in a small SQLite file, ~/.nova_harness/memory.db, and are loaded into the system prompt at the start of each conversation. The model saves them itself, and each save asks for your approval.

from NovaHarness.harness import Agent, MemoryStore

Agent("gpt-4o-mini")                                    # default store
Agent("gpt-4o-mini", memory=MemoryStore("memory.db"))   # your own file
Agent("gpt-4o-mini", memory=False)                      # none

Facts are capped at 50 entries and refuse new ones when full. Notes are capped at 50, evict the oldest, and only the 5 most recent reach the prompt.

What it guarantees

What it does not do. run_command is gated by your approval, not sandboxed: an approved command can reach anything your account can, so read it before you press y. Secret filtering is by name, a heuristic. fetch_url resolves a host and then connects, which leaves a DNS rebinding gap. A timed-out tool is abandoned, not killed, because Python cannot stop a thread.

The full security model is in the guide.

Reference

from NovaHarness.harness import Agent
from NovaHarness.tools import brave_search, ddgs_search

Agent("gpt-4o-mini", search_fn=ddgs_search())    # DuckDuckGo, no key
Agent("gpt-4o-mini", search_fn=brave_search())   # reads BRAVE_API_KEY

from NovaHarness.harness import MaxStepsReachedError, ModelError, PolicyAbortError

try:
    agent.run("Refactor everything.")
except PolicyAbortError:
    ...   # the model kept retrying a denied call
except MaxStepsReachedError:
    ...   # out of steps and no answer (final_answer=False, or the last call returned only tool calls)
except ModelError:
    ...   # the provider call failed

src/NovaHarness/
├── harness/   agent, runtime (the loop), executor, policies, approval, memory, messages, state, errors
├── models/    the ModelClient protocol and LiteLLMClient
└── tools/     @tool, ToolRegistry, built-in tools and the default policy
tests/         offline tests with scripted fake models and approvers

Development

pip install -e ".[dev]"
pytest

The suite needs no API key. To test your own tools, script a FakeModelClient (see tests/fakes.py) with the calls you want the "model" to make and assert on what comes back.

For the full guide (tool rules, every option, the security model in depth), see GUIDE.md.

License

MIT. Built by Marhfoul Bakr.