Frame-log
Overview
Frame-log is a framework aiming to make agents more aware of their environment through the explicit modeling of pre and post effects. The approach is inspired by logic-based AI theory such as the "AI Frame Problem".
Repository Overview
This repository implements the Frame-Log framework, providing utility methods that are usable across agentic workflows. Specifically, the following are included:
- Prompt templates to elicit change-logs
- Pre-defined change-log schema
- Currently supports JSON. Future work can include XML or git diff formats.
- Extraction methods to decompose a set of change-logs from an agent response into structured objects
- Evaluation and sanity check methods for change-logs, including:
- Schema validation
- Action verification
- Semantic similarity-based effect grounding via sentence embeddings
Setup
pip install framelog
Quick Start
from framelog.extract_log import to_dict
from framelog.evaluator import ChangeLogEvaluator, ChangeLogFailureType
evaluator = ChangeLogEvaluator()
# Parse a change log from an agent response
change_log = to_dict(response, evaluator=evaluator)
# Advance the evaluator timestep
evaluator.step()
# Inspect failure rates
print(evaluator.no_json_rate())
print(evaluator.invalid_action_rate())
print(evaluator.failure_breakdown())
Supported Benchmarks
License
MIT
Do you spot any bugs? Inefficiencies? Or have any general suggestions? Please create an issue or contact me via miller.j.nasir@gmail.com
Metadata
Release files for framelog 0.1.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| framelog-0.1.5.tar.gz | 6.6 kB | Without provenance | Details for framelog-0.1.5.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| framelog-0.1.5-py3-none-any.whl | Python 3 | none | any | Details for framelog-0.1.5-py3-none-any.whl |
Total release size: 16.0 kB
Release files / framelog-0.1.5.tar.gz
| Download URL | framelog-0.1.5.tar.gz |
|---|---|
| Size | 6.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
778816284ce60c761a167d2c04144eeb6925d75a5921bb3f94bd5916be0d49c0
|
|
BLAKE2b-256 checksum How to use checksums |
2b231082b85f40e727ea38fab37426e6f3f9e002d105351a25b243402c620020
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.15
|
Release files / framelog-0.1.5-py3-none-any.whl
| Download URL | framelog-0.1.5-py3-none-any.whl |
|---|---|
| Size | 9.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
e6a0209d391d415d186e275c05f8e9260126c3b8f69a58e32b598dab3e57fd46
|
|
BLAKE2b-256 checksum How to use checksums |
4b25f07384e3abb9bb8c90dc2ca03714af90c74dcc08b7147d703d4891d8ddd8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.15
|
Security for framelog 0.1.5
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.