Skip to main content

Jarvis-HEP-Sampler

Jarvis-HEP-Sampler is the plug-in SDK for user-defined samplers in the Jarvis ecosystem.

A user writes a sampler as one Python file, loads it once, and from then on uses it in a task card exactly like a built-in method. No Jarvis-HEP source code is changed. The plug-in's points go through the same Redis queue, logging and monitor as built-in samplers. The sampler-enabled HEP branch binds each scan to an exact source snapshot and checks scientific settings before resume. Project packages include the source and its full hash; fetch verifies it and prints a load command.

jsampler init MyLatinHypercube       # fixed template -> ./MyLatinHypercube.py
#   edit draw() (and optionally OPTIONS_SCHEMA)
jsampler check MyLatinHypercube.py   # optional offline check, no Redis
jsampler load MyLatinHypercube.py    # snapshot into ~/.jarvis/samplers; reloading replaces it
Jarvis run task.yaml                 # Sampling.Method: MyLatinHypercube

jsampler is this package's command for creating and checking sampler files and managing registrations in ~/.jarvis/samplers. Its list and info commands show registered external methods. Jarvis sampler is Jarvis-HEP's command and also knows the built-in methods. jsampler -v prints this package's version. jsampler --json prints one plain JSON object. The SDK also supports offline Python use (see Python API).

Scope

Jarvis-HEP-Sampler owns the plug-in format:

  • the fixed template
  • the PLUGIN metadata rules and method-name rule
  • loading a plug-in file
  • the user registry in ~/.jarvis/samplers/
  • the deterministic random streams
  • the output contract (shape, real, finite, inside [0, 1])
  • the log facade handed to plug-ins
  • the offline driver and conformance checks
  • the jsampler command

Jarvis-HEP owns the runtime:

  • Jarvis sampler, for built-in methods and loaded plug-ins
  • task-card validation
  • the run preflight
  • mapping unit-cube points to physical parameters, and selection
  • Sample UUIDs, Redis, Workers and the Archiver
  • checkpoints and resume
  • the sampler log, telemetry and monitor
  • Jarvis man and project packing

This package does not import Jarvis-HEP, Redis or Portal. jsampler check --card reads Sampling.Variables and Sampling.Bounds from the task YAML.

Complete package installation

Source candidate: Version 1.2.1. After publication, install from PyPI:

python -m pip install "Jarvis-HEP-Sampler==1.2.1"

From this repository:

python -m pip install .

Dependencies: NumPy, jsonschema, PyYAML and rich. Jarvis-HEP 2.1.0 and later require host contract 2, provided by Jarvis-HEP-Sampler 1.1.0 and later.

The end-to-end user tutorial lives in Jarvis-Books, in the TeX manual chapter Jarvis-HEP-Sampler: Implementing Your Own Sampler (TeX/Jarvis-HEP-V2 manual).

A plug-in

"""Latin hypercube design: one point per stratum in every variable."""

import numpy as np

PLUGIN = {
    "api_version": 1,
    "method": "LatinHypercube",
    "version": "0.1.0",
    "profile": "independent",
}

OPTIONS_SCHEMA = {  # optional
    "type": "object",
    "properties": {"centered": {"type": "boolean"}},
    "additionalProperties": False,
}


def prepare(setup):
    rng = setup.rng()
    perms = np.stack([rng.permutation(setup.point_number) for _ in range(setup.dimension)])
    return {"perms": perms, "centered": bool(setup.options.get("centered", False))}


def draw(context, rng, plan):
    strata = plan["perms"][:, context.candidate_index]
    offset = 0.5 if plan["centered"] else rng.random(context.dimension)
    return (strata + offset) / context.point_number
Sampling:
  Method: LatinHypercube
  Variables:
    - name: mass
      distribution: {type: Flat, parameters: {min: 100.0, max: 1000.0}}
  Bounds:
    seed: 7
    point_number: 10000
    centered: false

The full contract is in Plug-in authoring.

LatinHypercube is an external example in this SDK. Jarvis-HEP does not bundle it; load the file before using Sampling.Method: LatinHypercube. The algorithm and centered option are unchanged. Older SDK releases reserve this name; use the SDK 1.2.0 or newer.

A complete runnable card is provided in examples/LatinHypercube.task.yaml. It uses Jarvis-HEP's bundled Eggbox demonstration operator:

jsampler check examples/LatinHypercube.py --card examples/LatinHypercube.task.yaml
jsampler load examples/LatinHypercube.py
Jarvis validate examples/LatinHypercube.task.yaml
Jarvis check examples/LatinHypercube.task.yaml
Jarvis run examples/LatinHypercube.task.yaml

jsampler check accepts a sampler file only and does not need load first. It checks the method name before options schema, prepare or drawing points. A reserved-name failure returns exit 2; algorithm output or repeatability failures return exit 1. For a loaded method, use jsampler info <Method> to find its source file, then check that file. Jarvis check is the calculator smoke: its identity and bound source stay under DATABASE/test and checkpoints/<scan>/<Method>/check/, and the formal scan's science database and resume checkpoint stay in place.

The command-line layout, errors, exit codes and JSON format follow the Jarvis CLI design.

Reloading the same method updates future scans. Removing the registration preserves an existing scan's bound source and its normal Jarvis run ... --resume workflow.

Feedback MCMC with SliceMCMC

examples/SliceMCMC.py is a loadable generational plugin. The native Opera Eggbox card uses four chains, physical x/y in [0, 10*pi] radians and helper.eggbox2d, with z as LogLikelihood:

Jarvis sampler load examples/SliceMCMC.py
Jarvis validate examples/SliceMCMC.eggbox.task.yaml
Jarvis run examples/SliceMCMC.eggbox.task.yaml
# Continue the same scan after interruption:
Jarvis run examples/SliceMCMC.eggbox.task.yaml --resume

Use the HEP runtime with host contract 4 and SDK 1.2.1 or newer for the checkpoint example. The 20,000 global trial limit (Sampling.Bounds.max_evaluations) includes initialization, burn-in, rejected trials and selection rejects across all chains. The requested 1,000 retained transitions per chain may not finish before that limit. steps means retained transitions per chain; burn_in is discarded per chain. Evaluation failures halt by default. on_failure: reject explicitly treats failed regions as zero density.

Workers execute the Opera and evaluate Sampling.LogLikelihood in card order. An expression named LogL supplies the total. Otherwise the Worker sums the terms. tell receives the total as logL, with -inf for valid zero density and None for failed or selection-rejected evaluations. DATABASE/samples.csv contains the evaluation cloud, including rejected trials and burn-in. DATABASE/chain.csv contains posterior transitions only, with chain/step, generation, source UUID, unit/physical coordinates and logL. DATABASE/sampler_summary.json reports trials, Worker evaluations, selection rejects, failures, retained draws and the actual stop reason.

The SDK exports GenerationalSession, GenerationSetupContext and GenerationContext. Plugins implement init_state(setup), ask(state, context, rng) -> {state, proposals} | None, and tell(post_ask_state, context, ordered_results) -> state. The optional transitions(state, context) hook emits chain states; Metropolis plugins must emit repeated states for rejected transitions. State and tags are strict JSON.

jsampler init MyFeedbackSampler --profile generational
jsampler check examples/SliceMCMC.py --card examples/SliceMCMC.eggbox.task.yaml --points 200

check evaluates a toy Gaussian to check contracts, determinism and a fresh process restoring a barrier checkpoint. It does not execute card Operas or calculators. Jarvis run provides the real Worker feedback path. The direct python examples/SliceMCMC.py demonstration and the separate .offline.yaml runner remain optional local companions; the .task.yaml above is the native card.

sampler.log shows proposal-budget progress and an algorithm summary at startup, every 60 seconds and on exit. SliceMCMC reports initialization, discarded warmup and retained transitions per chain separately. The budget percentage does not measure convergence. Monitor readers use a cached snapshot.

Optional checkpoint support

Either profile can run with just its sampling functions. To support resume, define both hooks:

def checkpoint(state):
    return dict(state)

def resume(setup, saved):
    return saved

These hooks suit a state that already consists of supported data. For an independent sampler, the argument and return value are the plan passed to draw. For a generational sampler, they represent its committed algorithm state. Return all fields that affect future sampling. The dictionary can hold plain data and numerical NumPy arrays. The framework handles the cursor, source identity, file path and write schedule.

jsampler info shows checkpoint capability. jsampler check skips the resume check for plugins without hooks and verifies the resumed tail in a fresh process for plugins with hooks. HEP writes enabled checkpoints to runtime.pickle. Generation snapshots follow complete tell calls and reference the saved prefix of chain.csv. See the checkpoint contract.

Host integration

HOST_CONTRACT_VERSION = 4 provides optional checkpoint and resume hooks to the host; plug-in API_VERSION and the stream protocol remain 1. check_method_name and parse_metadata validate syntax and structure. External init, registry load and check_plugin apply check_external_method_name; generic loaders accept packaged names without a trusted-name override. Entry policies run through the optional metadata_validator loader hook before options schemas and hooks.

jarvis_hep_sampler.cli.main(..., catalog=CatalogExtension(...)) accepts host data callbacks augment_list(payload) and resolve_info(method, payload), plus list_sections=((payload_key, panel_title), ...). They add fields to the SDK results; parsing, errors and both text/JSON rendering stay in the SDK. Host info can include conflicts with exact method names, problems and removal commands. Existing JSON fields remain intact.

Python API

from jarvis_hep_sampler import SamplerRegistry, write_template
from jarvis_hep_sampler.testing import check_plugin, run_offline

path = write_template("MyLatinHypercube")                  # jsampler init
report = check_plugin(path, dimension=2, options={})     # jsampler check
for name, result in report.rows():
    print(f"{name:14} {result}")

run = run_offline(path, variable_names=["mass", "coupling"], point_number=1000, seed=7)
print(run.points.shape, run.stop_reason)

SamplerRegistry().load(path)                             # jsampler load

Documentation

Status

Alpha. API version 1 runs both independent and generational plugins. Either profile can opt in to checkpoint and resume through the two author hooks.

License

MIT. See LICENSE. Copyright (c) 2026 Pengxuan Zhu.

Releasing

jhsrel is a maintainer-local script, excluded from Git and distribution archives.

Record changes in CHANGELOG.md, then run from this checkout (replace the example version with the intended release):

./jhsrel 1.2.1 --dry
./jhsrel 1.2.1

--dry performs read-only source and release-note checks. The normal command requires a clean Git checkout, builds and checks the wheel and sdist, uploads to PyPI, refreshes the local version with python3 -m pip install -e ., and verifies the installed version. Only then does it commit the version, push the vX.Y.Z tag, and create a GitHub release with changelog notes. Configure Twine and gh authentication before publishing.

Local research

Maintainer experiments live under research/. The entire directory is ignored by Git and excluded from source distributions and wheels. Editable installation only installs the SDK under src/; research code is run explicitly from the checkout. Development notes live in Jarvis-Books.

Metadata

Release files for Jarvis-HEP-Sampler 1.2.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for Jarvis-HEP-Sampler 1.2.1
File Size Uploaded Provenance Actions
jarvis_hep_sampler-1.2.1.tar.gz 88.5 kB Without provenance Details for jarvis_hep_sampler-1.2.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for Jarvis-HEP-Sampler 1.2.1
File Interpreter ABI Platform Actions
jarvis_hep_sampler-1.2.1-py3-none-any.whl Python 3 none any Details for jarvis_hep_sampler-1.2.1-py3-none-any.whl

Total release size: 146.9 kB

Release files / jarvis_hep_sampler-1.2.1.tar.gz

Download URL jarvis_hep_sampler-1.2.1.tar.gz
Size 88.5 kB
Tags Source
SHA-256 checksum
How to use checksums
6491f7d618eca242c8fc489c74427a739b1d3ccb0e40b4dc2e4c6bc0e5a0223d
BLAKE2b-256 checksum
How to use checksums
ada480ee8838a7b2bfad41bd9d2374a572cba90962a614156581452442ea6465
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.13

Release files / jarvis_hep_sampler-1.2.1-py3-none-any.whl

Download URL jarvis_hep_sampler-1.2.1-py3-none-any.whl
Size 58.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
fadc3c38fc0f8b6baf91badd080339ec37bd3a5a112947435d48583ce16819e9
BLAKE2b-256 checksum
How to use checksums
1a8f88a375f839d35246d786b80fb167a9aff2e17542bf9e890159c84ca1f8d0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.13

Security for Jarvis-HEP-Sampler 1.2.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page