Skip to main content

EvalForge Maestro

Evaluate. Compare. Verify.

"EvalForge Maestro is an independent evaluation and regression verification engine — never an LLM runtime, RAG indexer, or agent framework."

EvalForge Maestro provides deterministic evaluation metrics, retrieval/RAG diagnostics, agent trace verification, OpenTelemetry span ingestion, statistical multi-trial cohort comparison, model-based judge calibration, and CI/CD quality gating behind a strict Pydantic v2 schema contract (evalforge-maestro CLI and evalforge_maestro Python SDK).


Key Architecture & Guarantees

  • Strict Domain Isolation: Zero coupling to document chunkers, runtime RAG stores, or agent orchestration loops. Target systems plug in via thin protocol adapters (CallableAdapter, RAGAdapter, lazy DocpipeAdapter, AgentAdapter) or completely offline (ReplayAdapter, ingest_otel_spans, load_otel_executions).
  • Cold-Start Purity: Importing evalforge_maestro or invoking evalforge-maestro --help never loads heavy or runtime-specific modules (litellm, docpipe_maestro, numpy, scipy, opentelemetry, opentelemetry_sdk, sqlite3, http.server) unless explicitly requested.
  • Deterministic & Immutable Artifacts: Canonical JSON hashing (SHA-256) of datasets and redacted configurations paired with atomic 0o700/0o600 run records under .evalforge/runs/ and append-only manifest.jsonl.
  • Defence-in-Depth Security: Pre-network --local-only endpoint validation (loopback-only enforcement and LITELLM_LOCAL_MODEL_COST_MAP), opt-in --block-pii egress scanning, prompt-injection XML/nonce delimiting in model-based judges, and automatic credential/API-key/Bearer/PII masking ([REDACTED]) across logs, traces, resolved configs, and reports.
  • Separation of Concerns: Every run and report strictly separates Evaluation (metric outcomes + retrieval-vs-answer/trace decoupling diagnostics), Regression (baseline cohort deltas, $p$-values, sliding OLS trends, anomaly flags, drift warnings), and Acceptance / Quality Gate (policy pass/fail verdict and POSIX exit code).

How EvalForge Maestro Works (Visual Guide)

EvalForge Maestro follows a strict, deterministic six-stage pipeline that separates executing your AI system from scoring quality, checking regressions, and enforcing release gates:

flowchart TD
    S1["1. Define Golden Dataset (JSONL / JSON / Docpipe Fixture)"]
    S2A["Live Target: CallableAdapter / RAGAdapter / DocpipeAdapter / AgentAdapter"]
    S2B["Offline Replay: ReplayAdapter (Captured JSONL or OpenTelemetry Spans)"]
    S3["3. Pre-Network Security Preflight (--local-only / --block-pii / Secret Scrubbing)"]
    S4["4. Bounded EvaluationOrchestrator + 4 Metric Families"]
    S5["5. Three-Way Engine: Evaluation | Regression (vs Baseline / Cohort) | QualityGate"]
    S6["6. Immutable Artifacts (.evalforge/runs/*.json + manifest.jsonl)"]
    S7["Optional Layers: SQLiteRunIndex (index/query) | HTTP Service (serve)"]

    S1 --> S2A
    S1 --> S2B
    S2A --> S3
    S2B --> S3
    S3 --> S4
    S4 --> S5
    S5 --> S6
    S6 --> S7

Step-by-Step Pipeline Walkthrough

Step What Happens & Why It Matters Concrete Components & Commands
1. Define Stream versioned test cases (EvaluationCase: input, reference, expected_context, expected_tool_calls, criteria.mandatory) and lock a canonical SHA-256 dataset_fingerprint so baseline comparisons never silently drift when test cases change. DatasetEngine, load_docpipe_fixture, --dataset
2. Adapt / Ingest Capture what your system actually produced (ExecutionResult: final output, retrieved_chunks, tool_calls, trace, tokens, latency). Run live against Python/RAG/Agent/Docpipe code or completely offline from pre-recorded executions or OTel spans. CallableAdapter, RAGAdapter, DocpipeAdapter, AgentAdapter, ReplayAdapter, ingest_otel_spans
3. Security Preflight Before any non-mock network call is attempted, --local-only rejects non-loopback judge/embedding endpoints and --block-pii scans payloads for emails, tokens, and credentials (exiting cleanly with POSIX 4). validate_local_only_target, validate_pii_egress, redact_secrets
4. Orchestrate & Score EvaluationOrchestrator runs cases under bounded concurrency (--max-concurrency), checkpoints per-case progress atomically, truncates oversized persisted traces after scoring (--max-context-chars), and evaluates up to 4 metric families simultaneously. Deterministic text/JSON, Retrieval IR (100% branch), Agent Trace (100% branch), LLM Judges & Dense Embeddings
5. Three-Way Evaluate & Gate Never collapses metrics into a single opaque average. Produces (a) raw Evaluation scores + decoupling diagnostics, (b) Regression deltas, Student's/Welch's $t$-test $p$-values (--repeats), OLS trend slopes, & anomaly flags (>0.3 spread), and (c) deterministic Quality Gate verdicts (PASS / FAIL / WARNING / INCONCLUSIVE). compare_runs, compare_run_cohorts, analyze_trend, QualityGate, --strict
6. Persist, Index & Serve Writes immutable 0o600 JSON RunRecord artifacts + append-only manifest.jsonl and pinned baselines.json. Optionally builds a rebuildable SQLite index (SQLiteRunIndex) for fast queries or exposes everything behind Bearer-authenticated HTTP (ServiceApplication / serve). LocalRunStore, SQLiteRunIndex, evalforge-maestro index / query / serve

Inside the Metric Families & Three-Way Diagnosis

Rather than hiding failures behind a macro average, EvalForge Maestro isolates where a RAG or agent pipeline broke:

flowchart LR
    EX["Captured ExecutionResult (output + retrieved_chunks + tool_calls/trace)"]

    M1["Deterministic Text & Schema (exact_match, normalized_match, json_schema_validation)"]
    M2["Retrieval IR Ranking (precision_at_k, recall_at_k, mrr, ndcg_at_k, hit_rate_at_k)"]
    M3["Agent Tool & Trace (tool_selection, argument_correctness, execution_sequence, efficiency, policy, recovery)"]
    M4["Semantic & Judge Scoring (--mock offline OR local/remote LiteLLM: groundedness, relevance, correctness, embeddings)"]

    DEC{"Three-Way & Decoupling Diagnostics"}
    G1["PASS / FAIL / WARNING / INCONCLUSIVE (POSIX Exit 0..4)"]

    EX --> M1
    EX --> M2
    EX --> M3
    EX --> M4
    M1 --> DEC
    M2 --> DEC
    M3 --> DEC
    M4 --> DEC
    DEC -->|"Retrieval OK + Generation Failed -> Fix Prompt / LLM"| G1
    DEC -->|"Retrieval Failed -> Fix Chunking / Hybrid / Reranker"| G1
    DEC -->|"Trace Violation / Policy Breach -> Fix Agent Guardrails"| G1

Calibrating Small & Local Judges First: Before letting a local model (such as Ollama/vLLM) gate pull requests, run evalforge-maestro calibrate --samples ... --metric groundedness --min-agreement 80.0 to measure exact_agreement_rate, directional bias (too_generous_rate vs. too_harsh_rate), mean_score_delta, and recommended_pass_threshold (docs/calibration-and-ci.md).

Minimal Core vs. Lazy Optional Extras

Core installation keeps dependencies minimal and cold-start instant; optional runtimes are only imported when invoked:

flowchart LR
    CORE["evalforge-maestro Core (pydantic + jsonschema only): CLI, Datasets, Deterministic/IR/Agent Metrics, Statistical t-tests, OTel Ingestion, SQLite Index, HTTP Service"]
    E1["[docpipe-maestro]: DocpipeAdapter (FTS5 hybrid + cross-encoder rerank + answer_detailed)"]
    E2["[judges]: LiteLLM Gateway (Groundedness, Answer/Context Relevance, Custom Rubric, Calibration)"]
    E3["[embeddings]: numpy + LiteLLM (EmbeddingSimilarityMetric, ContextEmbeddingSimilarityMetric)"]

    CORE -.->|"lazy import -> MissingExtraError if absent"| E1
    CORE -.->|"lazy import -> MissingExtraError if absent"| E2
    CORE -.->|"lazy import -> MissingExtraError if absent"| E3

Installation

Install core evaluation, deterministic/retrieval/agent metrics, offline replay, and zero-dependency OpenTelemetry ingestion:

pip install evalforge-maestro

Optional capability extras (lazy-loaded on demand; never required at import time):

# Docpipe Maestro retrieval and RAG target adapter
pip install "evalforge-maestro[docpipe-maestro]"

# LiteLLM-backed model-as-judge evaluation and calibration
pip install "evalforge-maestro[judges]"

# Dense embedding similarity metrics (numpy + litellm)
pip install "evalforge-maestro[embeddings]"

For contributor development using uv:

uv sync --extra dev

CLI Quickstart Reference

Both evalforge-maestro and evalforge_maestro console commands expose eleven subcommands: run, ingest-otel, index, query, compare, trend, calibrate, baseline, report, config, and serve.

1. Offline CI / Local Regression Evaluation

evalforge-maestro run \
  --config examples/ci/evalforge.toml \
  --mock \
  --local-only \
  --strict \
  --json \
  --json-out report.json \
  --markdown-out report.md

2. Multi-Trial Statistical Cohorts & Regression Gating

Evaluate stochastic LLM targets across N repeatable seeded runs and enforce Student's / Welch's $t$-test regression gates:

evalforge-maestro run \
  --config examples/ci/evalforge.toml \
  --repeats 3 \
  --seed 10 \
  --baseline release-v1 \
  --fail-on-significant-regression \
  --json

3. OpenTelemetry Trace Ingestion & Replay (ingest-otel / run --otel-traces)

Evaluate OTLP JSON envelopes, flat span arrays, or JSONL span streams directly (or normalize them into replayable ExecutionResult JSONL artifacts) with zero OTel SDK dependencies:

# Normalize an OTel span export into replayable ExecutionResult JSONL
evalforge-maestro ingest-otel examples/ci/otel-traces-sample.json \
  --dataset examples/ci/golden-sample.jsonl \
  --output replays.jsonl \
  --system-version agent-v2.4.0 \
  --json

# Evaluate OTel traces directly with agent + retrieval metrics and strict quality gating
evalforge-maestro run \
  --config examples/ci/evalforge-otel.toml \
  --strict \
  --max-context-chars 4000 \
  --json

See the full OpenTelemetry Trace Ingestion Guide for GenAI / OpenInference / Traceloop attribute precedence, tool call deduplication, clock-skew clamping, token anti-double-counting, and correlation rules.

4. Dense Embedding Similarity Evaluation

evalforge-maestro run \
  --config examples/ci/evalforge.toml \
  --metric embedding_similarity \
  --metric context_embedding_similarity:aggregation=mean \
  --mock \
  --local-only \
  --json
# Pin a trusted baseline without mutating immutable run artifacts
evalforge-maestro baseline set <RUN_ID> --name release-v1 --json

# Compare single candidate or multi-trial candidate cohort against baseline(s)
evalforge-maestro compare <CANDIDATE_RUN_ID> --baseline release-v1 --strict --json
evalforge-maestro compare <CAND_R01> <CAND_R02> \
  --baseline release-v1 --baseline-run <BASE_R02> \
  --fail-on-significant-regression --json

# Inspect sliding-window OLS metric trend slopes
evalforge-maestro trend --window 5 --json

# Calibrate a judge metric against labeled human samples with minimum agreement gate
evalforge-maestro calibrate \
  --samples examples/ci/calibration-samples.jsonl \
  --metric groundedness \
  --min-agreement 80.0 \
  --mock \
  --local-only \
  --json

# Render Three-Way Markdown/JSON diagnostic reports from stored artifacts
evalforge-maestro report --run <RUN_ID> --baseline release-v1 > diagnostic-report.md

# Audit effective resolved configuration with provenance tiers (secrets always masked)
evalforge-maestro config validate --config evalforge.toml --json
evalforge-maestro config show --resolved

6. Local SQLite Query Index (index & query)

Build or sync an optional derived SQLite query index (.evalforge/index.sqlite3) from canonical JSON artifacts and query runs or failing cases cleanly—deleting or corrupting the DB never loses data and rebuilds atomically:

# Inspect sync parity or atomically rebuild from .evalforge/runs/*.json + manifest.jsonl
evalforge-maestro index status --storage-dir .evalforge --json
evalforge-maestro index rebuild --storage-dir .evalforge --json

# Query historical runs and triage mandatory/failing cases
evalforge-maestro query runs --storage-dir .evalforge --dataset-id support-qa --metric exact_match --min-score 0.8 --json
evalforge-maestro query cases --storage-dir .evalforge --status FAIL --mandatory-only --json

See the Rebuildable SQLite Query Index Guide for full recovery guarantees, normalized DDL tables, and filter references.

7. Optional Remote Evaluation HTTP Service (serve)

Serve canonical storage queries, SQLite index lifecycle, OTel trace ingestion, offline/trace evaluation, comparison gates, and report rendering over a typed stdlib HTTP/1.1 JSON API (safe loopback default 127.0.0.1:8787, local_only=True, pre-bind remote auth guard, and constant-time Bearer authentication):

# Local loopback server in offline mock mode (defaults to local_only=True)
evalforge-maestro serve --storage-dir .evalforge --mock --log-level INFO

# Non-loopback host requires --auth-token or EVALFORGE_SERVICE_TOKEN (else exits 4 pre-bind)
EVALFORGE_SERVICE_TOKEN="strong-secret-token" evalforge-maestro serve \
  --host 0.0.0.0 \
  --port 8787 \
  --storage-dir .evalforge \
  --allow-remote-judges \
  --block-pii \
  --log-level INFO

See the Remote Evaluation Service Guide for security guarantees, complete route/schema tables, curl examples, and in-process ServiceApplication embedding.


Configuration Precedence, TOML Schema & Exit Codes

5-Tier Precedence Order

  1. CLI Flags (--dataset, --otel-traces, --strict, etc.)
  2. OS Environment Variables (EVALFORGE_*, e.g. EVALFORGE_JUDGE_API_KEY, EVALFORGE_OTEL_TRACES, EVALFORGE_OTEL_CASE_ID_ATTR)
  3. Opt-in Environment File (--env-file <path>)
  4. Configuration File (--config <path> or non-recursive ./evalforge.toml in CWD)
  5. Built-in Safe Defaults (mock_mode=false, storage_dir=".evalforge", max_concurrency=4, strict security guards)

TOML configuration validates strictly (extra="forbid") across [run], [target], [otel], [metrics], [judge], [embedding], [security], [gate], and [logging] ([judge] and [embedding] credentials/endpoints remain strictly isolated).

Output Streams

  • --json: Emits strictly one schema-validated, secret-redacted JSON payload on stdout; diagnostics, warnings, and --log-json / --log-level logs write exclusively to stderr.
  • Default (non---json): Writes structured Markdown summary reports to stdout (suppressible via --quiet). --json-out and --markdown-out write report files concurrently.

POSIX Exit Codes

Exit Code Meaning Typical Causes
0 Success Gate PASS (or WARNING/INCONCLUSIVE when --strict is disabled), diagnostic calibrate, ingest-otel, baseline, report, or config.
1 Quality Gate Failure Metric threshold breach, mandatory case failure, regression drop violation, significant regression, or --strict escalating WARNING/INCONCLUSIVE.
2 Input / Config / Alignment Error Invalid CLI flags, unknown evalforge.toml keys, missing file, corrupt JSON/JSONL, replay alignment mismatch, conflicting --otel-traces + --executions/--target, or unresolved OTel case_id.
3 Runtime / Extra Failure Adapter or judge execution fault under --fail-fast, zero cases completed, or missing optional extra (MissingExtraError).
4 Security / Policy Violation Pre-network block triggered by --local-only non-loopback endpoint or --block-pii PII egress detection.

Python SDK Quickstarts

RAG & Deterministic Evaluation

from evalforge_maestro import (
    DatasetEngine,
    EvaluationOrchestrator,
    PrecisionAtKMetric,
    RAGAdapter,
    RecallAtKMetric,
    exact_match,
)

dataset = DatasetEngine(
    [
        {
            "id": "q1",
            "input": "What is the capital of France?",
            "reference": "Paris",
            "expected_context": ["doc-paris"],
        }
    ]
)

adapter = RAGAdapter(
    lambda q: {
        "answer": "Paris",
        "chunks": [{"id": "doc-paris", "text": "Paris is the capital of France."}],
    }
)

run = EvaluationOrchestrator().run(
    dataset,
    adapter,
    [exact_match, PrecisionAtKMetric(k=1), RecallAtKMetric(k=1)],
)
print(run.aggregates["recall_at_k"].mean_score)

OpenTelemetry Trace Ingestion & Agent Replay

from evalforge_maestro import (
    ArgumentCorrectnessMetric,
    DatasetEngine,
    EvaluationOrchestrator,
    OtelExtractorConfig,
    ReplayAdapter,
    ToolSelectionMetric,
    ingest_otel_spans,
    load_otel_executions,
)

# 1. Normalize in-memory OTLP dicts, flat span lists, or duck-typed SDK ReadableSpan objects
executions = ingest_otel_spans(otlp_payload)

# 2. Or load from disk (OTLP JSON / array JSON / JSONL) into ReplayAdapter for orchestrator evaluation
dataset = DatasetEngine("examples/ci/golden-sample.jsonl")
adapter = ReplayAdapter.from_otel_file(
    "examples/ci/otel-traces-sample.json",
    extractors=OtelExtractorConfig(match_dataset_by_input=True),
    dataset=dataset,
)

run = EvaluationOrchestrator().run(
    dataset,
    adapter,
    [ToolSelectionMetric(), ArgumentCorrectnessMetric()],
)
print(run.aggregates["tool_selection"].mean_score)

Rebuildable SQLite Run & Case Querying

from evalforge_maestro import CaseQueryFilter, MetricStatus, RunQueryFilter, SQLiteRunIndex

index = SQLiteRunIndex(".evalforge")
runs = index.query_runs(
    RunQueryFilter(dataset_id="support-qa", metric_id="exact_match", min_mean_score=0.8)
)
failed = index.query_cases(CaseQueryFilter(status=MetricStatus.FAIL, mandatory_only=True))

In-Process HTTP Service Dispatcher (ServiceApplication)

import json

from evalforge_maestro import ServiceApplication, ServiceConfig

app = ServiceApplication(
    ServiceConfig(storage_dir=".evalforge", local_only=True, mock_mode=True),
    auth_token="service-secret",
)
status, headers, body = app.handle_request(
    "GET",
    "/health",
    headers={"Authorization": "Bearer service-secret"},
)
assert status == 200 and json.loads(body.decode("utf-8"))["status"] == "ok"

Documentation & CI Templates


Implemented Milestones (M1–M6 + Post-M6 Extensions)

  • M1 — Evaluation Foundation (001-evaluation-foundation): Single Pydantic v2 schema source (evalforge_maestro.types), lazy JSONL streaming DatasetEngine, CallableAdapter, deterministic text/JSON Schema metrics (exact_match, normalized_match, json_schema_validation), bounded async EvaluationOrchestrator, and atomic 0o700/0o600 .evalforge/ storage.
  • M2 — Retrieval & RAG Evaluation (002-retrieval-rag-evaluation): Protocol-based RAGAdapter, lazy DocpipeAdapter ([docpipe-maestro]), dependency-free fixture importer, and versioned IR metrics (precision_at_k, recall_at_k, mrr, ndcg_at_k, hit_rate_at_k) with AST formula verification (100% branch coverage).
  • M3 — Experiment & Regression Engine (003-experiment-regression-engine): Pinned baseline aliases, pairwise aggregate/per-case comparisons, sliding-window OLS trend slopes (analyze_trend), anti-masking/disagreement anomalies (> 0.3 divergence), provenance drift alerts, and deterministic QualityGate (100% branch coverage).
  • M4 — Model-Based Judge & Calibration Engine (004-judge-calibration-engine): Lazy LiteLLM gateway ([judges]), pre-network --local-only & LITELLM_LOCAL_MODEL_COST_MAP air-gap safety, randomized XML/nonce prompt-injection delimiting, offline --mock scoring, context-grouped bounded async batching, and human calibration (calibrate_judge / CalibrationReport).
  • M5 — CLI, Reporting & CI Integration (005-cli-reporting-ci): Dual CLI entrypoints (evalforge-maestro / evalforge_maestro), offline ReplayAdapter, opt-in --block-pii egress guard, library-pure evalforge_maestro.reporting (ReportDocument JSON & Three-Way Markdown separating Evaluation, Regression, and Quality Gate + retrieval-vs-answer decoupling diagnostics), and copy-ready CI workflow templates.
  • M6 — Agent Trace Evaluation (006-agent-trace-evaluation): Backward-compatible typed trace/tool contracts (ExpectedToolCall, ToolCallRecord, AgentTraceStep), six 100%-branch-covered deterministic agent metrics (tool_selection, argument_correctness, execution_sequence, tool_efficiency, policy_compliance, failure_recovery), dependency-free AgentAdapter + structural OpenAI/Anthropic trace normalizers, CLI --target-type agent support, field-level post-scoring max_context_chars trace truncation, and decoupled trace-vs-answer diagnostics.
  • Statistical & Embedding Metrics (007-statistical-embedding-metrics): Lazy [embeddings] extra (numpy + litellm) powering EmbeddingSimilarityMetric and ContextEmbeddingSimilarityMetric with strict [judge]/[embedding] isolation, SHA-256 --mock fallback, pre-network --local-only / --block-pii checks, pure-stdlib Student's $t$ / Welch's $t$ / one-sample $t$ confidence intervals and $p$-values (compare_runs, compare_run_cohorts, build_pooled_cohort_run), --repeats immutable cohorts, and QualityGate.evaluate_cohort (fail_on_significant_regression).
  • OpenTelemetry Execution Trace Ingestion (008-otel-trace-ingestion): Dependency-free ingestion (ingest_otel_spans, load_otel_executions, ReplayAdapter.from_otel_file, OtelExtractorConfig, OtelTimestampUnit) normalizing OTLP JSON, flat span arrays, JSONL spans, and duck-typed SDK ReadableSpan objects across GenAI/OpenInference/Traceloop conventions into secret-scrubbed canonical ExecutionResult traces, plus [otel] / [run].otel_traces config precedence, evalforge-maestro run --otel-traces, and evalforge-maestro ingest-otel.
  • Rebuildable Local SQLite Query Index (009-sqlite-query-index): Zero-dependency SQLiteRunIndex (evalforge_maestro.storage) backed atomically by canonical .evalforge/runs/*.json, manifest.jsonl, and baselines.json with 0o700/0o600 permissions, PRAGMA journal_mode=DELETE, transparent deletion/corruption recovery, parameterized query_runs / query_cases / metric_history, and CLI evalforge-maestro index & query subcommands.
  • Optional Remote Evaluation Service & Consumer CI Guides (010-remote-eval-service): Library-pure ServiceApplication (evalforge_maestro.service), lazy stdlib ThreadingHTTPServer (create_http_server, serve_http), and CLI evalforge-maestro serve enforcing pre-bind remote auth policies (validate_service_bind_policy), constant-time Bearer auth, local_only=True defaults, HTTP 413 body limits, secret scrubbing, and zero http.server cold-start leakage—paired with verified hybrid/rerank (evalforge-hybrid-rerank.toml) & small-judge calibration (calibration-samples.jsonl) CI fixtures and comprehensive guides (docs/remote-service.md, docs/calibration-and-ci.md).

CI/CD & PyPI Trusted Publishing Setup

  • CI Pipeline (.github/workflows/ci.yml): Enforces uv lock --check, ruff format --check, ruff check, Bandit S checks, mypy --strict, >=85% global branch coverage, 100% branch coverage on evalforge_maestro.metrics and evalforge_maestro.regression.gate, @pytest.mark.gate_determinism / @pytest.mark.gate_immutability, pip-audit, cz check, cross-OS testing (ubuntu-latest, macos-latest, windows-latest), uv build + twine check --strict, clean-venv wheel/sdist installs + CLI cold-start smoke checks, and --no-dev MissingExtraError isolation.
  • Release Pipeline (.github/workflows/release.yml): Triggered on v* tags; re-runs all verification gates, verifies hatch-vcs wheel version matches the Git tag, publishes via OIDC Trusted Publishing (id-token: write) to TestPyPI, validates clean wheel + CLI entrypoint installation from TestPyPI, promotes to PyPI, and creates the GitHub Release.
    • One-time repository prerequisite before pushing the first v* tag: configure GitHub environments named testpypi and pypi and register matching OIDC Trusted Publishers on test.pypi.org and pypi.org targeting .github/workflows/release.yml.

Metadata

Release files for evalforge-maestro 0.3.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for evalforge-maestro 0.3.0
File Size Uploaded Provenance Actions
evalforge_maestro-0.3.0.tar.gz 1.2 MB With provenance Details for evalforge_maestro-0.3.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for evalforge-maestro 0.3.0
File Interpreter ABI Platform Actions
evalforge_maestro-0.3.0-py3-none-any.whl Python 3 none any Details for evalforge_maestro-0.3.0-py3-none-any.whl

Total release size: 1.4 MB

Release files / evalforge_maestro-0.3.0.tar.gz

Download URL evalforge_maestro-0.3.0.tar.gz
Size 1.2 MB
Tags Source
SHA-256 checksum
How to use checksums
3459d330f6f00e69c96d921089fac5e6ebc156cf8004ab48f797c4e744564973
BLAKE2b-256 checksum
How to use checksums
7ef830e3046a31dc2d71aadeda5d6488de38027ece7258d170ef3f7bf075d019
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 21, 2026.

Transparency log

Release files / evalforge_maestro-0.3.0-py3-none-any.whl

Download URL evalforge_maestro-0.3.0-py3-none-any.whl
Size 204.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
3ff1b3bc54bab6f5f8bcb42bab1406e2f0e6d1e03b675b327411899f7d8ef8ec
BLAKE2b-256 checksum
How to use checksums
1bdced7ae63aaf23172dacd661f19170abf7c0264b1f03dd9ca1bbf86be55b4f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 21, 2026.

Transparency log

Security for evalforge-maestro 0.3.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for evalforge_maestro-0.3.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 21, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page