pyforla

License: MIT Python

CLI client for LFORLA — run LLM benchmarks locally and push results to the leaderboard. Pairs with the LFORLA platform.

Features

  • pull — download benchmark metadata and dataset samples (JSON/YAML)
  • run — evaluate samples locally against an LLM API (OpenAI, Anthropic, Ollama, generic)
  • push — submit results to the LFORLA leaderboard
  • report — generate a markdown/HTML evaluation report from results
  • dataset upload — upload sample files to MinIO and link them to a benchmark
  • bench-push — publish benchmark definitions (moderator-only)

Install

pip install pyforla

Or from source:

git clone https://github.com/lforla/pyforla.git
cd pyforla && pip install -e .

Verify:

lforla-eval --help

Quick Start

# Save your API key (create one in your LFORLA account settings)
lforla-eval login "your-api-key"

# Download a benchmark's samples
lforla-eval pull root-cause-analysis -o ./data

# Run the samples locally against a model
lforla-eval run ./data/root-cause-analysis_samples.json -m gpt-4o -p openai

# Push the results to the leaderboard
lforla-eval push results.json --benchmark-id <uuid>

Commands

Command Description
lforla-eval login <api-key> Save your API key to ~/.config/lforla/config.json
lforla-eval whoami Check authentication status
lforla-eval list-benchmarks List available benchmarks
lforla-eval pull <slug> [-o dir] Download benchmark metadata + samples
lforla-eval run <samples> -m <model> [-p provider] Run samples against a model
lforla-eval push <results> --benchmark-id <uuid> Submit results to LFORLA
lforla-eval report <results> [-o report.md] Generate an evaluation report
lforla-eval dataset upload <file> <slug> Upload a sample file and create a dataset version
lforla-eval bench-push <file.yaml> Publish a benchmark definition (moderator-only)

Docker

A container image is provided for isolated or server-side runs:

docker build -t lforla-eval .
docker run --rm -v "$PWD:/data" lforla-eval run /data/samples.json -m gpt-4o -p openai

Documentation

License

MIT

Project links

Data verified by PyPI on Aug 14, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for resk from gravatar.com resk

Credits

Author: LFORLA

GitHub Statistics

Data verified by PyPI on Aug 14, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

MIT
View SPDX License List

Requires

Python >=3.10

Tags

benchmark cli evaluation lforla llm

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pyforla-0.3.0.tar.gz (11.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

pyforla-0.3.0-py3-none-any.whl (12.5 kB view details)

Uploaded Python 3

File details

Details for the file pyforla-0.3.0.tar.gz.

File metadata

  • Download URL: pyforla-0.3.0.tar.gz
  • Upload date:
  • Size: 11.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for pyforla-0.3.0.tar.gz
Algorithm Hash digest
SHA256 35fb08f6281aa08a579ce6fa8179adc45388a3fdcdcb404b567f1bc4052a598e
MD5 faaa8cc42abe73ff0335ad43f5e8478f
BLAKE2b-256 1564c05dd81916a4927ae734a48d2b3a2dc8a553df11268f5d6883167b8ebe17

See more details on using hashes here.

Provenance

The following attestation bundles were made for pyforla-0.3.0.tar.gz:

Publisher: publish-pypi.yml on Lforla/pyforla

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file pyforla-0.3.0-py3-none-any.whl.

File metadata

  • Download URL: pyforla-0.3.0-py3-none-any.whl
  • Upload date:
  • Size: 12.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for pyforla-0.3.0-py3-none-any.whl
Algorithm Hash digest
SHA256 9d40b2d71fd0f882d09f114ff538f3e5c2e6b2dc948cfcd7930b2d19a28f532b
MD5 c652cbd5df078c8e6a865ced8888bde4
BLAKE2b-256 949f65ad95249a665b3b5668afcd7b8c39565086647518f58692491b632b9ca5

See more details on using hashes here.

Provenance

The following attestation bundles were made for pyforla-0.3.0-py3-none-any.whl:

Publisher: publish-pypi.yml on Lforla/pyforla

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.3.0 This release

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page