Skip to main content

valor-lite: Fast, local machine learning evaluation.

valor-lite is a lightweight, numpy-based library designed for fast and seamless evaluation of machine learning models. It is optimized for environments where quick, responsive evaluations are essential, whether as part of a larger service or embedded within user-facing tools.

valor-lite is maintained by Striveworks, a cutting-edge MLOps company based in Austin, Texas. If you'd like to learn more or have questions, we invite you to connect with us on Slack or explore our GitHub repository.

For additional details, be sure to check out our user documentation. We're excited to support you in making the most of Valor!

Usage

Classification

from valor_lite.classification import DataLoader, Classification, MetricType

classifications = [
    Classification(
        uid="uid0",
        groundtruth="dog",
        predictions=["dog", "cat", "bird"],
        scores=[0.75, 0.2, 0.05],
    ),
    Classification(
        uid="uid1",
        groundtruth="cat",
        predictions=["dog", "cat", "bird"],
        scores=[0.41, 0.39, 0.1],
    ),
]

loader = DataLoader()
loader.add_data(classifications)
evaluator = loader.finalize()

metrics = evaluator.evaluate()

assert metrics[MetricType.Precision][0].to_dict() == {
    'type': 'Precision',
    'value': [0.5],
    'parameters': {
        'score_thresholds': [0.0],
        'hardmax': True,
        'label': 'dog'
    }
}

Object Detection

from valor_lite.object_detection import DataLoader, Detection, BoundingBox, MetricType

detections = [
    Detection(
        uid="uid0",
        groundtruths=[
            BoundingBox(
                xmin=0, xmax=10,
                ymin=0, ymax=10,
                labels=["dog"]
            ),
            BoundingBox(
                xmin=20, xmax=30,
                ymin=20, ymax=30,
                labels=["cat"]
            ),
        ],
        predictions=[
            BoundingBox(
                xmin=1, xmax=11,
                ymin=1, ymax=11,
                labels=["dog", "cat", "bird"],
                scores=[0.85, 0.1, 0.05]
            ),
            BoundingBox(
                xmin=21, xmax=31,
                ymin=21, ymax=31,
                labels=["dog", "cat", "bird"],
                scores=[0.34, 0.33, 0.33]
            ),
        ],
    ),
]

loader = DataLoader()
loader.add_bounding_boxes(detections)
evaluator = loader.finalize()

metrics = evaluator.evaluate()

assert metrics[MetricType.Precision][0].to_dict() == {
    'type': 'Precision',
    'value': 0.5,
    'parameters': {
        'iou_threshold': 0.5,
        'score_threshold': 0.5,
        'label': 'dog'
    }
}

Semantic Segmentation

Pixel value 0 is background; positive value i refers to labels[i - 1]. Provide foreground names only. Background counts toward accuracy, not mean IoU.

import numpy as np
from valor_lite.semantic_segmentation import Loader, MetricType, Segmentation

segmentation = Segmentation(
    uid="image-1",
    labels=["sky", "road", "car"],
    groundtruths=np.array([[0, 1], [2, 3]], dtype=np.uint16),
    predictions=np.array([[0, 1], [1, 3]], dtype=np.uint16),
)

loader = Loader.in_memory()
loader.add_data([segmentation])
evaluator = loader.finalize()
metrics = evaluator.compute_precision_recall_iou()
print(metrics[MetricType.Precision][0])

Metadata

Release files for valor-lite 0.38.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for valor-lite 0.38.0
File Size Uploaded Provenance Actions
valor_lite-0.38.0.tar.gz 77.9 kB Without provenance Details for valor_lite-0.38.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for valor-lite 0.38.0
File Interpreter ABI Platform Actions
valor_lite-0.38.0-py3-none-any.whl Python 3 none any Details for valor_lite-0.38.0-py3-none-any.whl

Total release size: 172.1 kB

Release files / valor_lite-0.38.0.tar.gz

Download URL valor_lite-0.38.0.tar.gz
Size 77.9 kB
Tags Source
SHA-256 checksum
How to use checksums
3c1e1061289a5f26fa2271143e5199a17f024837c24b799d5b96eeacdc1935c2
BLAKE2b-256 checksum
How to use checksums
f8f89e92fa5d99a10475896829b8c10b12c4b315a041087ac50429eedd53e275
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / valor_lite-0.38.0-py3-none-any.whl

Download URL valor_lite-0.38.0-py3-none-any.whl
Size 94.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
57d0a9fc787a8035aef111e7e912627df1d6d91f66ab8f0d005ae92067724e1f
BLAKE2b-256 checksum
How to use checksums
598ad05298ddb63a871936f58b9c047cd8ce4eb83c77b19070374ff4ad5b52d7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Security for valor-lite 0.38.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.38.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page