SAGE Tool Use Benchmark
Configuration-driven experiment framework for evaluating tool selection and use capabilities.
Features
- Tool Selection Evaluation: Tool retrieval and ranking benchmarks
- Planning Evaluation: Multi-step planning with tool composition
- Timing Detection: Timing judgment for tool invocation decisions
Quick Start
# Install
pip install isage-tooluse-benchmark
# Run tool selection experiment
sage-tooluse-bench tool-selection --config config/tool_selection_exp.yaml
# Run planning experiment
sage-tooluse-bench planning --config config/planning_exp.yaml
Documentation
See benchmark_agent/README.md for detailed documentation.
Development
# Clone
git clone https://github.com/intellistream/sage-tooluse-benchmark.git
cd sage-tooluse-benchmark
# Setup virtual environment
python -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
# Install in development mode
pip install -e ".[dev]"
# Run tests
pytest
License
MIT License - see LICENSE for details.
Metadata
Release files for isage-tooluse-benchmark 0.1.0.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| isage_tooluse_benchmark-0.1.0.1.tar.gz | 124.5 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| isage_tooluse_benchmark-0.1.0.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 268.9 kB
Release files / isage_tooluse_benchmark-0.1.0.1.tar.gz
| Download URL | isage_tooluse_benchmark-0.1.0.1.tar.gz |
|---|---|
| Size | 124.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
bfc8be65a36200196c088f3ce40e924a199b2efc0e2455c7d1bd13f69e6935b8
|
|
BLAKE2b-256 checksum How to use checksums |
98e017d69629b63a3460366a6f9d25d9a3d7727962d21c01d78b55cb93e25470
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.11
|
Release files / isage_tooluse_benchmark-0.1.0.1-py3-none-any.whl
| Download URL | isage_tooluse_benchmark-0.1.0.1-py3-none-any.whl |
|---|---|
| Size | 144.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
61d7f65e84f7396517c817615101298d39c60fc89bd9cdf3073d5cf4f799537a
|
|
BLAKE2b-256 checksum How to use checksums |
0a6924e0827237d24a7fe361ff75a9aa174c564907cedd7a57a07d283aef3400
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.11
|
