Skip to main content

Jarvis-Operas

Jarvis-Operas is a standalone operator layer for registering, loading, and calling Python callables.

  • Scope: operators only (likelihood, chi2, prior mapping, data transforms, etc.)
  • No dependency on Jarvis-HEP internals
  • Native async entrypoint for Jarvis-HEP style execution (await registry.acall(...))
  • Stable "<namespace>.<name>" naming for references from Jarvis-HEP/Jarvis-PLOT YAML

Documentation lives in Jarvis-Books:

Install

pip install .

Or for development (includes pytest and installs terminal command):

pip install -e ".[dev]"

Terminal command (jopera)

After install, you can use:

jopera
jopera help examples
jopera namespaces
jopera list --namespace math
jopera list --namespace math --json
jopera info stat.chi2_cov --json
jopera call math.add --kwargs '{"a": 1, "b": 2}'
jopera acall stat.chi2_cov --arg residual=[1.0,-0.5] --arg cov=[[2.0,0.1],[0.1,1.0]]
jopera call helper.eggbox --kwargs '{"x":0.5,"y":0.0}'
jopera call helper.eggbox --arg x=90 --arg y=0 --input-units '{"x":"deg","y":"rad"}' --unit-audit --json
jopera call pdg.mZ --output-units '{"":"MeV"}' --json
jopera list --namespace math --log-mode info
jopera call math.add --kwargs '{"a": 1, "b": 2}' --log-mode debug
jopera init
jopera init --manifest ./manifest.json --cache-root ~/.jarvis/operas/curve-cache
jopera interp list
jopera interp validate
jopera --help-advanced

Load user operators directly in CLI:

jopera load /absolute/path/to/my_ops.py
jopera call my_ops.my_op --user-ops /absolute/path/to/my_ops.py --arg x=10
jopera info my_ops.my_op --json

CLI behavior:

  • jopera (no args) prints a quick start card
  • jopera list prints grouped human-readable output by default (id + name)
  • jopera list --json prints machine-readable JSON objects with id/name/namespace
  • jopera info <name_or_id> prints metadata (including a unique id) and a suggested next command
  • call / acall accept --input-units, --source-units, --output-units, --strict-units, and --unit-audit; unit options return the canonical result, projected values, units, normalized inputs, and audit
  • jopera load <path> persists a JO-managed source snapshot for future processes
  • Re-running jopera load <path> replaces existing user operators with the same <namespace>.<name>
  • Removing a function from a user source file does not auto-delete its prior registration
  • jopera load <path> --session-only loads without persisting
  • jopera init precompiles bundled interpolation manifest library
  • jopera init --manifest <manifest.json> precompiles a custom curve JSON manifest
  • jopera interp list lists root interpolation namespaces
  • jopera interp validate validates root+namespace manifests
  • User write policy: the supported user write path is jopera load

Core API

Physical operators can declare a versioned metadata.unit_contract. Calls without source units retain canonical-unit defaults; call_units / acall_units return the original result, separately projected values and a per-call audit. Prepare reusable plans with registry.prepare(...) during Worker preload. For example:

from jarvis_operas import get_global_operas_registry

registry = get_global_operas_registry()
result = registry.call_units("pdg.mZ", output_units={"": "MeV"})
assert result.values[""] == 91187.6

Use jopera call pdg.mZ --output-units '{"":"MeV"}' --json from the CLI. The unit guide includes the metadata schema, strict source validation, TeV → GeV input, and pb → fb output examples. The JSON Schema describes declaration structure; registered unit symbols and callable parameters are validated by JO. HEP YAML unit fields and its executor bridge remain downstream work.

from jarvis_operas import get_global_operas_registry, get_global_operas

registry = get_global_operas_registry()
registry.call("math.add", a=1, b=2)
registry.call("math.identity", x={"k": 1})

operas = get_global_operas()
operas.call("math.add", a=1, b=2)

Import the same operators directly in a script. The module path is joperas.<namespace>, and the imported name is the operator's short name:

from joperas.math import add
from joperas.pdg import mZ
from joperas.dmddxn import LZSI2024
from joperas.dmddxe import CDEXLM2022

add(1, 2)          # 3
mZ()               # 91.1876 GeV
LZSI2024(30.0)     # DM-nucleon curve, x in GeV, result in cm^2

mZ() is a zero-argument call because PDG constants are registered operators. DM-nucleon curves live in dmddxn and DM-electron curves live in dmddxe. Every name returned by registry.list() can be imported this way, including a user operator after it is loaded into the global registry. A persisted operator file can use the same import at module level. Importing joperas publishes the registry first, and the namespace import only reads that registry. Startup registers built-in operators and curves before it executes the file. The imported object is the NumPy implementation. When the operator declares a unit contract, the call normalizes inputs to canonical units. A name that is already bound keeps that function; import it again after a registry replacement to see the new one. registry.call(..., backend="polars") remains the Polars path. A non-string backend keyword is passed through to the operator on both the imported function and registry.call.

Supported namespace convention:

  • Built-ins use feature namespaces, e.g.:
    • math.<name>
    • stat.<name>
    • helper.<name>
  • User operator files default to <script_name>.<func_name>
  • User operators cannot register into protected built-in namespaces (for example math, stat, helper, interp, built-in interpolation namespaces).

Async call (Jarvis-HEP Factory/Module friendly)

result = await registry.acall(
    "stat.chi2_cov",
    residual=[1.0, -0.5],
    cov=[[2.0, 0.1], [0.1, 1.0]],
)

Behavior:

  • Async operator: awaited directly
  • Sync operator: offloaded via asyncio.to_thread by default (or custom executor)
  • Default numpy async concurrency is CPU-scaled (os.cpu_count() * 3); override with get_global_operas(numpy_concurrency=...)
  • Batch helper concurrency: await registry.acall_helper_many("eggbox", [...])

Example batch helper execution for Factory-side concurrent scans:

results = await registry.acall_helper_many(
    "eggbox",
    [
        {"x": 0.1, "y": 0.2},
        {"x": 0.3, "y": 0.4},
        {"x": 0.5, "y": 0.6},
    ],
)

Logger injection

registry methods accept optional logger and operators can optionally define logger argument.

  • If no logger is provided, Jarvis-Operas uses loguru.logger bound with module="Jarvis-Operas"
  • If logger is provided, Jarvis-Operas reuses it (no duplicate handler creation)
  • Console format follows Jarvis-HEP style (module -> time - [level] >>> message)
  • Default mode is warning (only warning/error/critical are shown)
  • Optional modes: info, debug

Utility:

from jarvis_operas import get_logger, set_log_mode

set_log_mode("info")   # or "debug", default is "warning"
logger = get_logger()

Load user operators from file

from jarvis_operas import OperasRegistry, load_user_ops

registry = OperasRegistry()
loaded = load_user_ops("./my_ops.py", registry)
print(loaded)

Reload behavior:

  • load_user_ops(...) and jopera load always execute the current source file contents
  • If a user operator resolves to an existing <namespace>.<name>, the new definition replaces the old one
  • If a function is removed from the file, JO does not auto-prune the old registration; delete/rename remains an explicit override action

Persistent registration for future Python processes:

from jarvis_operas import get_global_operas_registry, persist_user_ops

persist_user_ops("/absolute/path/to/my_ops.py")
registry = get_global_operas_registry()  # auto-loads persisted sources

Persistence store location:

  • Jarvis home is ~/.jarvis (JARVIS_HOME). Operas state lives in operas/ (JARVIS_OPERAS_HOME). ~/.jarvis-operas/ is not used.
  • Default sources registry: ~/.jarvis/operas/sources.json
  • Default override rules: ~/.jarvis/operas/overrides.json
  • Legacy base env (still supported): JARVIS_OPERAS_PERSIST_FILE=/custom/path/persist_store.json
    • maps to sibling files sources.json and overrides.json
  • Optional explicit envs:
    • JARVIS_OPERAS_SOURCES_FILE=/custom/path/sources.json
    • JARVIS_OPERAS_OVERRIDES_FILE=/custom/path/overrides.json
  • Managed user source snapshots: ~/.jarvis/operas/user_sources/
  • Override snapshot dir: JARVIS_OPERAS_USER_SOURCE_DIR=/custom/path/user_sources
  • Legacy single-file user_ops.json is still readable for backward compatibility

By default, load_user_ops("./my_ops.py", ...) uses my_ops as namespace, so my_op becomes my_ops.my_op.

my_ops.py can export operators with either style:

  1. Decorator (recommended)
from jarvis_operas import oper

@oper("my_chi2")
def my_chi2(residual, cov, logger=None):
    ...

When loaded by load_user_ops("./my_ops.py", ...), this is registered as my_ops.my_chi2.

  1. Explicit whitelist
def my_op(x):
    return x

__JARVIS_OPERAS__ = {
    "my_op": my_op,
}

Integration Contract (Jarvis-HEP / Jarvis-PLOT)

Import integration entry points from jarvis_operas: OperasRegistry, Operas, get_global_operas_registry, get_global_operas, oper, load_user_ops, and persist_user_ops. Downstream modules should use the public registry methods:

  • registry provider: get_global_operas_registry()
  • runtime calls: call/acall
  • lookup/introspection: resolve_name/get/list/info
  • unit execution: prepare/call_units/acall_units/acall_many_units

Prepared plans expose run/arun, run_inputs/arun_inputs, and project. Prepare registrations and fixed unit plans during host initialization, then reuse them during execution. Direct implementation calls bypass unit validation; downstream integrations must use these public entry points. JO provides declarations, registration, and dispatch; hosts own sample objects, YAML parsing, and observable storage. The user guide links to current unit execution examples and archived migration notes.

Function documentation generation

Function docs are generated from OperaFunction.metadata:

python scripts/generate_function_docs.py --out docs/functions

Generated artifacts are local outputs under the ignored docs/ directory. The published copy is the function reference in Jarvis-Books.

  • docs/functions/index.md
  • docs/functions/<namespace>/index.md
  • docs/functions/functions.json
  • docs/functions/<namespace>/<function>.md

Optional filters and output directory:

python scripts/generate_function_docs.py --namespaces math,stat
python scripts/generate_function_docs.py --out docs/custom_functions

Curve publish/runtime cache (namespace manifests + JSON sources)

Use this flow when you have many 1D interpolation curves and want runtime speed:

  1. Source of truth:
    • root index manifest (interpolations.manifest.json) with namespace list
    • one manifest per namespace (function metadata list)
    • per-function JSON source (x/y arrays)
  2. Precompile once: jopera init (bundled library) or jopera init --manifest ./manifest.json (custom)
    • optional: jopera init --namespaces dmddxn,interp1 for partial namespace processing
    • an init updates the namespaces in that manifest and leaves every other namespace already stored in the same cache. Startup initialization of the bundled library does this too, so a custom manifest compiled into the default cache remains available
    • inspect manifests with jopera interp list and jopera interp validate
  3. Runtime auto-registration: get_global_operas_registry() initializes the bundled cache and registers hot curves in their resolved namespaces, such as dmddxn.LZSI2024 and dmddxe.CDEXLM2022
  4. After initialization, cached curve loading uses index.json plus numeric .npz artifacts. Loading rebuilds the interpolator and does not execute pickle. The default cache directory is ~/.jarvis/operas/curve-cache (JARVIS_HOME or JARVIS_OPERAS_HOME overrides the parent).

Namespace rule for registered interpolation operators:

  • Use the first available declaration: the curve item's namespace, its group, its metadata.group, then the namespace manifest or root-index namespace.
  • Register as <resolved_namespace>.<curve_id>; use interp.<curve_id> only when no namespace is declared.
  • Bundled interpolation namespaces are interp1, dmddxn (DM-nucleon), and dmddxe (DM-electron).

Bundled interpolation manifest library resource:

  • jarvis_operas/manifests/interpolations.manifest.json

Root index example:

{
  "version": 2,
  "kind": "jarvis_operas_interpolation_namespace_index",
  "namespaces": [
    {
      "namespace": "demo",
      "manifest": "demo/demo.manifest.json"
    }
  ]
}

Namespace manifest example:

{
  "namespace": "demo",
  "functions": [
    {
      "name": "demo_curve",
      "source": "demo/demo_curve.json",
      "kind": "linear",
      "hot": true,
      "metadata": {
        "group": "demo"
      }
    }
  ]
}

Curve source JSON example:

{
  "x": [0.0, 1.0, 2.0],
  "y": [0.0, 1.0, 4.0]
}

Python integration API:

from jarvis_operas import (
    init_curve_cache,
    interpolation_manifest_resource,
    load_hot_curve_function_table,
    load_interpolation_manifest_library,
    register_hot_curves,
)

library_manifest = load_interpolation_manifest_library()
library_path = interpolation_manifest_resource()

init_curve_cache("./manifest.json")
table = load_hot_curve_function_table()

funcs = {}
updated = register_hot_curves(funcs)

Interpolation functions support:

  • numpy backend: native callable
  • polars backend: map_batches fallback via OperaFunction.return_dtype
  • sync: registry.call(...)
  • async: await registry.acall(...)

Built-in operators

Namespace Available operators
math Generic add and identity.
stat Covariance chi-square with chi2_cov.
helper Benchmark functions eggbox and eggbox2d.
pdg Particle masses, couplings, and physical constants, including mZ, GF, and hbarc.
cmb tt_cosmopower prediction and loglike on the bundled 111-point Planck TT grid.
interp1 Four general interpolation examples.
dmddxn 27 DM-nucleon exclusion curves; input in GeV and output in cm².
dmddxe 20 DM-electron exclusion curves; input in MeV and output in cm².

Use jopera namespaces, jopera list --namespace NAME, and jopera info NAME to discover functions and their signatures, units, and supported backends. Sync/async dispatch is available through the registry; accepted containers and backend support depend on each function. Examples:

registry.call("math.add", a=1.0, b=2.0)
registry.call("stat.chi2_cov", residual=[1.0, 0.0], cov=[[2.0, 0.0], [0.0, 1.0]])
registry.call("helper.eggbox", x=0.5, y=0.0)

Query registry for external UIs (JHEP/JPlot)

registry.list()
registry.list(namespace="math")
registry.info("stat.chi2_cov")

registry.info(...) returns metadata, signature, docstring summary, backend support fields, and async flag.

Releases

jorel is a maintainer-local script, excluded from Git and distribution archives.

Record the release changes under Unreleased in CHANGELOG.md. Use the next version number (shown here as 1.6.0):

./jorel 1.6.0 --dry
./jorel 1.6.0

--dry performs read-only source and release-note checks. The normal command requires a clean Git checkout, builds and checks the wheel and sdist, runs the release tests, uploads to PyPI, refreshes this checkout with python3 -m pip install -e ., and verifies the installed version. Only then does it commit the version, push the vX.Y.Z tag, and create a GitHub release from the changelog. Configure Twine and gh authentication before publishing. PyPI versions cannot be overwritten.

Release changes are recorded in CHANGELOG.md.

Workspace clean

Clean local cache/build artifacts:

./scripts/clean_workspace.sh

Metadata

Release files for Jarvis-Operas 1.6.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for Jarvis-Operas 1.6.0
File Size Uploaded Provenance Actions
jarvis_operas-1.6.0.tar.gz 8.2 MB Without provenance Details for jarvis_operas-1.6.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for Jarvis-Operas 1.6.0
File Interpreter ABI Platform Actions
jarvis_operas-1.6.0-py3-none-any.whl Python 3 none any Details for jarvis_operas-1.6.0-py3-none-any.whl

Total release size: 16.4 MB

Release files / jarvis_operas-1.6.0.tar.gz

Download URL jarvis_operas-1.6.0.tar.gz
Size 8.2 MB
Tags Source
SHA-256 checksum
How to use checksums
55ca842b263edffd6b31643cf505c5a526c7acaa963f0cc6a64bbd739e64f053
BLAKE2b-256 checksum
How to use checksums
72a6258ef690db8ec31c2270cc2c0c0d2644116c9e7394619469f7aded752604
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.13

Release files / jarvis_operas-1.6.0-py3-none-any.whl

Download URL jarvis_operas-1.6.0-py3-none-any.whl
Size 8.2 MB
Tags Python 3
SHA-256 checksum
How to use checksums
731f924bfdfb4bfa456c19f0696b1f125fb6419e5649a37d9498b666c81da644
BLAKE2b-256 checksum
How to use checksums
f50c06a2f869028f9988ae1b369d683f72e7b301281dbe2bd04f5b9e86f09d34
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.13

Security for Jarvis-Operas 1.6.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page