aieng-eval-agents
Shared library for Vector Institute's Agentic AI Evaluation Bootcamp. Provides reusable components for building, running, and evaluating LLM agents with Google ADK and Langfuse.
What's included
Agent implementations
| Module | Description |
|---|---|
aieng.agent_evals.knowledge_qa |
ReAct agent that answers questions using live web search. Includes evaluation against the DeepSearchQA benchmark with LLM-as-a-judge metrics (precision/recall/F1). |
aieng.agent_evals.aml_investigation |
Agent that investigates Anti-Money Laundering cases by querying a SQLite database of financial transactions via a read-only SQL tool. |
aieng.agent_evals.report_generation |
Agent that generates structured Excel reports from a relational database based on natural language queries. |
aieng.agent_evals.misalignment_qa |
Config-driven experiment runner for measuring LLM misalignment under varying context conditions, with YAML-defined variants, LLM-as-judge scoring, and Langfuse trace analysis. |
Reusable tools (aieng.agent_evals.tools)
search— Google Search with response grounding and citationsweb— HTML and PDF content fetchingfile— Download and search data files (CSV, XLSX, JSON)sql_database— Read-only SQL database access viaReadOnlySqlDatabase
Evaluation harness (aieng.agent_evals.evaluation)
Wrappers around Langfuse for running agent experiments:
run_experiment— Run a dataset through an agent and score outputsrun_experiment_with_trace_evals— Run experiments with trace-level evaluationrun_trace_evaluations— Score existing Langfuse traces with LLM-based or heuristic graders
Utilities
display— Rich-based terminal and Jupyter display helpers for metrics and agent responsesprogress— Progress tracking for batch evaluation runsconfigs— Pydantic-based configuration loading from.envlangfuse— Langfuse client and trace utilitiesdb_manager— Database connection management
Installation
pip install aieng-eval-agents
Requires Python 3.12+.
Source
Full reference implementations and documentation are in the eval-agents repository.
1 maintainer
Metadata
1 maintainer
Release files for aieng-eval-agents 0.3.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| aieng_eval_agents-0.3.2.tar.gz | 178.6 kB | With provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| aieng_eval_agents-0.3.2-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 408.8 kB
Release files / aieng_eval_agents-0.3.2.tar.gz
| Download URL | aieng_eval_agents-0.3.2.tar.gz |
|---|---|
| Size | 178.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
9326202a1b99b31bf841667bcd027b7edf6228d57915c1d82a0a76fa935d4e5f
|
|
BLAKE2b-256 checksum How to use checksums |
4ad9a40a756253e92a6a72f0d2575b70565075949811aeadcae56a77c9dacd48
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logRelease files / aieng_eval_agents-0.3.2-py3-none-any.whl
| Download URL | aieng_eval_agents-0.3.2-py3-none-any.whl |
|---|---|
| Size | 230.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
c3720824ae2bad76f6a06d22610a631f65cc8a190834adb790f1357c14de39f8
|
|
BLAKE2b-256 checksum How to use checksums |
a2a717aa0b9bc0acdffdffb2ce07bdd85426d05f7566cc11d1e3e352c5885d8f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logSecurity for aieng-eval-agents 0.3.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
All 2 files in this release have been uploaded with provenance metadata.
Attestation for aieng_eval_agents-0.3.2.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
