selfevolve

Most "self-improving AI" means retraining weights. This is the other half: the agent improves the system around the model — its rules, memory, prompt, workflow — and it is not allowed to ship a change until a back-test proves the change is better.

v0.1 implements the smallest honest version of that loop over one evolvable surface: rules.

measure → collect misses → mine candidates → back-test gate → promote or log rejection → re-measure

Install

pip install -e .

The fraud example

The screener starts with one deliberately naive rule: flag anything over $800. It runs against labelled transactions, keeps every fraud it missed, and mines candidate rules from those misses. Each candidate is back-tested; it is promoted only if F1 actually improves by a margin, and every rejection is logged with its reason.

selfevolve-fraud

One cycle, measured on a held-out 40% the evolution never saw:

107 candidates were mined. 3 promoted, 104 rejected. That ratio is the point — the agent proposes freely and ships almost nothing.

Promoted rules:

amount >= 800                              (the original)
velocity_1h >= 7                           (card testing)
new_device AND country_mismatch            (account takeover)
amount >= 222 AND new_device

Using it on your own data

from selfevolve import Policy, Rule, Condition, evolve_once, score

policy = Policy([Rule((Condition("amount", ">=", 800.0),))])
report = evolve_once(policy, train, holdout, fields=["amount", "velocity_1h", "new_device"])

print(report.before, "->", report.after)
for v in report.verdicts:
    print(v)          # every promotion AND every rejection, with the reason

Records are plain dicts with a boolean label (default key is_fraud, configurable). No dependencies.

The guardrail

gate() is the whole safety story. A candidate must clear two bars:

• support — it has to fire on at least min_support true positives, so the agent can't overfit to one weird transaction.
• gain — it has to lift F1 by at least min_gain on the back-test set.

Everything else is rejected and recorded. Evolution happens on train; the numbers you report come from holdout, which the gate never touches.

What else an agent can evolve

v0.1 moves rules. The same measure-gate-promote loop applies to every other surface:

• Memory — what it stores from past runs, and what it pulls back in next time.
• Prompt — its own instructions get stricter or more specific after repeated failures.
• Examples — which few-shot cases it shows itself for a given task type.
• Retrieval — how it chunks, ranks and decides what's worth fetching.
• Knowledge — the underlying corpus and index: adding, correcting, retiring facts.
• Tools — which tool it picks, how it describes them, and sometimes writing new ones.
• Skills — reusable procedures it saves once and adapts later.
• Workflow — the steps themselves: adding verify, critique, retry, or a second search pass.
• Routing — which model or sub-agent handles which kind of task.
• Roles — spawning new specialist agents and how work is delegated between them.
• Stopping rules — when to give up, retry, escalate to a human, or spend more budget.
• Evaluator — the critic and rubric it judges itself against (the riskiest one to let move).
• Weights — optionally, at the far end: LoRA, DPO or fine-tuning on its own successful traces.

License

MIT