Skip to main content

valor-lite: Fast, local machine learning evaluation.

valor-lite is a lightweight, numpy-based library designed for fast and seamless evaluation of machine learning models. It is optimized for environments where quick, responsive evaluations are essential, whether as part of a larger service or embedded within user-facing tools.

valor-lite is maintained by Striveworks, a cutting-edge MLOps company based in Austin, Texas. If you'd like to learn more or have questions, we invite you to connect with us on Slack or explore our GitHub repository.

For additional details, be sure to check out our user documentation. We're excited to support you in making the most of Valor!

Usage

Classification

from valor_lite.classification import DataLoader, Classification, MetricType

classifications = [
    Classification(
        uid="uid0",
        groundtruth="dog",
        predictions=["dog", "cat", "bird"],
        scores=[0.75, 0.2, 0.05],
    ),
    Classification(
        uid="uid1",
        groundtruth="cat",
        predictions=["dog", "cat", "bird"],
        scores=[0.41, 0.39, 0.1],
    ),
]

loader = DataLoader()
loader.add_data(classifications)
evaluator = loader.finalize()

metrics = evaluator.evaluate()

assert metrics[MetricType.Precision][0].to_dict() == {
    'type': 'Precision',
    'value': [0.5],
    'parameters': {
        'score_thresholds': [0.0],
        'hardmax': True,
        'label': 'dog'
    }
}

Object Detection

from valor_lite.object_detection import DataLoader, Detection, BoundingBox, MetricType

detections = [
    Detection(
        uid="uid0",
        groundtruths=[
            BoundingBox(
                xmin=0, xmax=10,
                ymin=0, ymax=10,
                labels=["dog"]
            ),
            BoundingBox(
                xmin=20, xmax=30,
                ymin=20, ymax=30,
                labels=["cat"]
            ),
        ],
        predictions=[
            BoundingBox(
                xmin=1, xmax=11,
                ymin=1, ymax=11,
                labels=["dog", "cat", "bird"],
                scores=[0.85, 0.1, 0.05]
            ),
            BoundingBox(
                xmin=21, xmax=31,
                ymin=21, ymax=31,
                labels=["dog", "cat", "bird"],
                scores=[0.34, 0.33, 0.33]
            ),
        ],
    ),
]

loader = DataLoader()
loader.add_bounding_boxes(detections)
evaluator = loader.finalize()

metrics = evaluator.evaluate()

assert metrics[MetricType.Precision][0].to_dict() == {
    'type': 'Precision',
    'value': 0.5,
    'parameters': {
        'iou_threshold': 0.5,
        'score_threshold': 0.5,
        'label': 'dog'
    }
}

Semantic Segmentation

Pixel value 0 is background; positive value i refers to labels[i - 1]. Provide foreground names only. Background counts toward accuracy, not mean IoU.

import numpy as np
from valor_lite.semantic_segmentation import Loader, MetricType, Segmentation

segmentation = Segmentation(
    uid="image-1",
    labels=["sky", "road", "car"],
    groundtruths=np.array([[0, 1], [2, 3]], dtype=np.uint16),
    predictions=np.array([[0, 1], [1, 3]], dtype=np.uint16),
)

loader = Loader.in_memory()
loader.add_data([segmentation])
evaluator = loader.finalize()
metrics = evaluator.compute_precision_recall_iou()
print(metrics[MetricType.Precision][0])

Metadata

Release files for valor-lite 0.39.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for valor-lite 0.39.0
File Size Uploaded Provenance Actions
valor_lite-0.39.0.tar.gz 78.3 kB Without provenance Details for valor_lite-0.39.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for valor-lite 0.39.0
File Interpreter ABI Platform Actions
valor_lite-0.39.0-py3-none-any.whl Python 3 none any Details for valor_lite-0.39.0-py3-none-any.whl

Total release size: 173.0 kB

Release files / valor_lite-0.39.0.tar.gz

Download URL valor_lite-0.39.0.tar.gz
Size 78.3 kB
Tags Source
SHA-256 checksum
How to use checksums
2ea85ace1fe1474d6caacb8b452dd76e3c9355b7f5a5598589a997028dc006cd
BLAKE2b-256 checksum
How to use checksums
fa155e2ecdead48ab200fa5598ec34966ebe1d0d2fba1f764ea1e2c7b4f54967
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / valor_lite-0.39.0-py3-none-any.whl

Download URL valor_lite-0.39.0-py3-none-any.whl
Size 94.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
578fce9b077a957d08e8606afc96bc4de484ab3e3239b8674780a98c6405cb05
BLAKE2b-256 checksum
How to use checksums
d51d2eecd1d56c832c1b0e7938c9b080ed2d64a631438eabb278d0a01901a2c1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Security for valor-lite 0.39.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.39.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page