This release is a pre-release and may not be stable for production use.
Nullius
Bring anything. Run it. Prove it.
Nullius runs AI evaluations on your machines and opens each score into the evidence behind it: what failed, the agent or the benchmark.
Install and launch
Install the latest version with uv:
uv tool install nullius-eval
nullius serve
Open 127.0.0.1:5057. The package name is nullius-eval;
the application and command are nullius.
Python 3.12 or later is required. Reading existing results needs no model key. Running benchmarks requires the appropriate provider credentials and benchmark prerequisites; Docker is needed for Harbor and the standard's docker backend.
This is a release candidate for testing before version 0.1.0. Release
validation targets Linux with Python 3.12. The wheel is a Python package with
a local web UI, not a standalone desktop installer.
What it does
- Bring benchmarks, agents and capabilities into a catalogue and match them by the ports they have proven.
- Build evaluation recipes, review their prerequisites, then run locally, over SSH or across several machines.
- Inspect trial outcomes, verifier results, model calls, conversations and artifacts; compare the shared tasks across runs.
- Read Harbor job directories and import ATIF or OpenTelemetry traces.
- Check benchmarks with controls and inspect results with optional model judgement.
Settings, catalogue additions and runs are stored outside the installed
package. nullius paths shows their locations, and reinstalling the tool
preserves them.
Updating
Update to the latest version with:
uv tool upgrade nullius-eval
To select a specific version, use uv tool install nullius-eval==0.1.0rc1.
Add --force when replacing an existing installation. Explicit version pins
are retained by uv tool upgrade; use uv tool install --force nullius-eval
to return to the latest version without a pin.
If you used an earlier private build installed under the distribution name
nullius, remove that tool with uv tool uninstall nullius before installing
nullius-eval. The user storage remains in place.
Links
GitHub access follows the repository's permissions. Nullius is licensed under Apache-2.0; bundled fonts retain their own included licences.
1 maintainer
Metadata
1 maintainer
Release files for nullius-eval 0.1.0rc1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| nullius_eval-0.1.0rc1.tar.gz | 1.1 MB | Without provenance | Details for nullius_eval-0.1.0rc1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| nullius_eval-0.1.0rc1-py3-none-any.whl | Python 3 | none | any | Details for nullius_eval-0.1.0rc1-py3-none-any.whl |
Total release size: 2.4 MB
Release files / nullius_eval-0.1.0rc1.tar.gz
| Download URL | nullius_eval-0.1.0rc1.tar.gz |
|---|---|
| Size | 1.1 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ec16f0bec27ae9bd76a1a9f958042e6a0f73fbeadb43863dc8fc0a90c777a970
|
|
BLAKE2b-256 checksum How to use checksums |
7b25d67f6e17a285f81349e262f4f077828ed420e9a4965b6c0c44fd16301e7c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.13.1 {"installer":{"name":"uv","version":"0.13.1","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / nullius_eval-0.1.0rc1-py3-none-any.whl
| Download URL | nullius_eval-0.1.0rc1-py3-none-any.whl |
|---|---|
| Size | 1.3 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
01150f60e85d9be338a4e984c7bedcf0c78f3c95a131dfa30375fce89724383d
|
|
BLAKE2b-256 checksum How to use checksums |
6b27ab3c9d90bcbb42eef88dfccea14ec2f647efb8fac6b08740e458aa27bef0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.13.1 {"installer":{"name":"uv","version":"0.13.1","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Security for nullius-eval 0.1.0rc1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
