Agent Behavior Specification (ABS)

A vendor-neutral, human-readable format for describing the observable behavior of AI agents — what users say, what agents do, and how it should be evaluated. Like OpenAPI for HTTP APIs, ABS gives agent behavior a shared, tool-independent contract.

📖 Full documentation · 📦 GitHub

Install

pip install abslang

Commands

abslang init

Scaffold a new ABS project with an example session and dataset.

abslang init

Creates abs.config.yaml, sessions/order-status.abs.yaml, and sessions/order-status.jsonl (3 rows).

abslang run

Execute ABS sessions against an agent.

# Single session
abslang run sessions/order-status.abs.yaml --agent http://localhost:8080/chat

# With a dataset (parametrized testing — one run per row)
abslang run sessions/order-status.abs.yaml --agent $URL --dataset sessions/order-status.jsonl

# With a single variable override
abslang run sessions/order-status.abs.yaml --agent $URL --var orderId=12345

# All sessions in a directory
abslang run sessions/ --agent $URL --dataset datasets/

# CI mode with JUnit output
abslang run sessions/ --agent $STAGING --dataset datasets/ --format junit --ci > report.xml
Option Description
--agent <url> Agent endpoint URL (or set ABS_AGENT_URL)
--dataset <path> JSON/JSONL dataset file
--var key=value Single variable binding (repeatable)
--filter key:value Filter dataset rows
--agent-format openai (default), claude, or gemini
--agent-auth none, api_key, bearer, or oauth2
--agent-token Auth token or API key
--adapter llm_judge=aievaluator Route LLM judge through AI Evaluator
--format table (default), json, or junit
--ci CI mode (no colors)
--timeout <n> Timeout per session in seconds (default: 300)
--output <path> Write report to file
--parallel <n> Run N dataset rows in parallel

abslang report

View results from a previous abslang run --output.

abslang report report.json                  # Table view
abslang report report.json --format json    # Machine-readable
abslang report report.json --format junit   # CI integration
abslang report report.json --failed         # Only failed cases
abslang report report.json --detail 3       # Full trace for row #3

abslang chat

Generate ABS YAML by describing the behavior in plain language (requires DeepSeek API key).

export DEEPSEEK_API_KEY=sk-...
abslang chat

# You: A customer asks for a refund. The agent should verify the order, process it, and confirm.
# → generates .abs.yaml with evaluations, datasets, and chain checks

Commands inside chat: /save <path>, /force <path>, /quit.

abslang generate-ci

Generate a CI/CD workflow file.

abslang generate-ci --platform github   # GitHub Actions
abslang generate-ci --platform gitlab   # GitLab CI

Test with the mock agent

# Terminal 1: start mock agent
python tools/mock_agent.py --scenario happy

# Terminal 2: run the example
abslang run examples/order-status.yaml --agent http://localhost:8080/chat

Library usage

from abslang import parse, run
from abslang.runner import AgentConfig
import asyncio

session = parse('session.abs.yaml')
result = asyncio.run(run(session, AgentConfig(
    url='http://localhost:8080/chat',
    format='openai',
)))
print(result.passed)  # True | False

Links

License

Apache 2.0

Project links

Data verified by PyPI on
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.1.4)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for fvinciarelli from gravatar.com fvinciarelli

Credits

Author: Franco Vinciarelli

GitHub Statistics

Data verified by PyPI on 
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

Apache-2.0

Requires

Python >=3.10

Provides Extra

ai

Tags

agent ai behavior evaluation llm openapi specification testing

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

abslang-0.1.1.tar.gz (30.3 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

abslang-0.1.1-py3-none-any.whl (34.2 kB view details)

Uploaded Python 3

File details

Details for the file abslang-0.1.1.tar.gz.

File metadata

  • Download URL: abslang-0.1.1.tar.gz
  • Upload date:
  • Size: 30.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for abslang-0.1.1.tar.gz
Algorithm Hash digest
SHA256 34a62d598b8f1ac1b824afc051881018ea2b616c8f6539dd5d5b16c9a968ac46
MD5 92cfbef7768e6ad59610ba00ac3804fc
BLAKE2b-256 e98a85d93b5df475203eedae4fe63804b0b16bdd7dcc622677f013859eebf65b

See more details on using hashes here.

Provenance

The following attestation bundles were made for abslang-0.1.1.tar.gz:

Publisher: publish-pypi.yml on fvinciarelli/abslang

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file abslang-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: abslang-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 34.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for abslang-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 9ed7f9b87be8167627634c4b243e069df3c6adaf3d2f08f55c4bcbfbe10f333d
MD5 17854cc501d4b1b8bd2950d7ea68e1bc
BLAKE2b-256 02712e3f20e9c75568ba58739cd37e1e423c99d2c6246670876e2209bc4bb871

See more details on using hashes here.

Provenance

The following attestation bundles were made for abslang-0.1.1-py3-none-any.whl:

Publisher: publish-pypi.yml on fvinciarelli/abslang

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page