A Metropolis-Hastings sampler for LLMs.
Implements the sampling algorithm described in Reasoning with Sampling: Your Base Model is Smarter Than You Think in vLLM. This cuts the evaluation time for Qwen/Qwen2.5-Math-7B from 30hrs to <1hr on the MATH500 dataset on a B200 GPU compared to the official implementation.
This package patches the vLLM LLMEngine object and adds the alpha parameter to SamplingParams in order to sample from the power distribuion.
Installation
pip install mh-llm
or from the source:
pip install git+https://github.com/maxzuo/mh-llm.git
Example Usage
from mh_llm import MHLLM
from mh_llm.vllm import SamplingParams
# Initialize MH LLM with your model
mh_llm = MHLLM(model='Qwen/Qwen2.5-Math-7B')
# Define sampling parameters with alpha
sampling_params = SamplingParams(temperature=0.25, alpha=0.4)
# Generate samples, without metropolis-hastings or power distribution
output = mh_llm.generate("What is 1234 + 5678?", sampling_params=sampling_params)
# Sample with Metropolis-Hastings against the power distribution
mh_output = mh_llm.mh_sample(
"What is 1234 + 5678?",
sampling_params=sampling_params,
block_size=192,
max_new_tokens=3_072,
num_mcmc_steps=10,
use_tqdm=True,
)
1 maintainer
Metadata
1 maintainer
Release files for mh-llm 0.0.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| mh_llm-0.0.1.tar.gz | 15.7 kB | With provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| mh_llm-0.0.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 35.2 kB
Release files / mh_llm-0.0.1.tar.gz
| Download URL | mh_llm-0.0.1.tar.gz |
|---|---|
| Size | 15.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
785ae37a78e259d96f0de87c3d6326b400fb6c755e35aa45d32e8b0d54f7fbfc
|
|
BLAKE2b-256 checksum How to use checksums |
bc6940fbf90c9cacd58d26b635b5d90d23f6abec7dc1a827f245075182692820
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Dec 16, 2025.
Transparency logRelease files / mh_llm-0.0.1-py3-none-any.whl
| Download URL | mh_llm-0.0.1-py3-none-any.whl |
|---|---|
| Size | 19.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
c09507ffae6f83146cfff9cca1460c63adf01a337e3b5827f3825c28742bd4fa
|
|
BLAKE2b-256 checksum How to use checksums |
5563f0d3a3bd6c97c78ce59e0227c439588455cd1726a9900e50f07517749dfc
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Dec 16, 2025.
Transparency logSecurity for mh-llm 0.0.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
All 2 files in this release have been uploaded with provenance metadata.
Attestation for mh_llm-0.0.1.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Dec 16, 2025.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
