june-bench

A pip-installable, reproducible benchmark suite for memory / QA systems — June + pluggable competitors — over LoCoMo, LongMemEval, HotpotQA/2Wiki/MuSiQue, and FinanceBench, with the same data and the same scorer.

pip install june-bench
june-bench list
june-bench run --system echo --dataset smoke --split smoke    # offline, no key, no download

A benchmark is run(system, dataset) → records → score. Two typed ports are the only extension points:

  • System — the thing benchmarked. JuneApiSystem (default; a thin HTTP client to June's /v1/answer, so no June source is shipped), JuneLocalSystem ([june-local] extra; a source-protected compiled wheel), CogneeSystem ([cognee] extra), or any future system as one adapter.
  • Dataset — what it runs on. The four benchmarks behind a registry.

The scorer is the canonical SQuAD/HotpotQA EM/F1 + selective-accuracy/coverage/cost — Cognee-comparable. Tiny smoke fixtures ship in the wheel (offline wiring proof); full splits are fetched, sha-verified, from a pinned release. No score is ever baked into the package — every result row records dataset + scorer + system + model + cost, so a published number is reproducible by a stranger.

Status: SB0 (contracts + no-deps smoke + skeleton). Datasets, June/Cognee systems, and the full CLI land in SB1–SB6.

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.1.2)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for Bhuone2345 from gravatar.com Bhuone2345

Credits

Author: Junemind

License

MIT License (MIT)

Requires

Python >=3.10

Provides Extra

june-api june-local cognee stream all dev

Tags

benchmark rag memory qa retrieval evaluation llm june

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

june_bench-0.0.18.tar.gz (2.0 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

june_bench-0.0.18-py3-none-any.whl (2.1 MB view details)

Uploaded Python 3

File details

Details for the file june_bench-0.0.18.tar.gz.

File metadata

  • Download URL: june_bench-0.0.18.tar.gz
  • Upload date:
  • Size: 2.0 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for june_bench-0.0.18.tar.gz
Algorithm Hash digest
SHA256 954046acf8fd89df8c6e826e5fd78f9c46d6c37bf137cefb0f82a0b5cea9082c
MD5 11ae7d0d6acea2b5027a5afb81a69b2a
BLAKE2b-256 5713e078be062a3af07d95d382f9e4c33ded34a827070ef453e9a3765e42e2b5

See more details on using hashes here.

Provenance

The following attestation bundles were made for june_bench-0.0.18.tar.gz:

Publisher: publish-bench.yml on Junemind/june-brain

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file june_bench-0.0.18-py3-none-any.whl.

File metadata

  • Download URL: june_bench-0.0.18-py3-none-any.whl
  • Upload date:
  • Size: 2.1 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for june_bench-0.0.18-py3-none-any.whl
Algorithm Hash digest
SHA256 6f179bd3423c28f3d159238e2c3c962d8e8d0af660374bc9246a6241c700458b
MD5 06abed5fd1083e27697a6fca5a5759d8
BLAKE2b-256 7fbed57de58bb5d4ee16602edfab08d4ae1a35a63f221b55e49657fa16f5c3c5

See more details on using hashes here.

Provenance

The following attestation bundles were made for june_bench-0.0.18-py3-none-any.whl:

Publisher: publish-bench.yml on Junemind/june-brain

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page