Beam Search over Efficient llama.cpp generations
Theory
Forthcoming...
Example Usage
from llama_cpp_beamsearch.completion import BeamSearchCompletion
from llama_cpp_beamsearch.config import BeamSearchConfig
from llama_cpp import Llama, LlamaGrammar, ChatCompletionRequestMessage
model = Llama.from_pretrained(
repo_id="unsloth/Qwen3-0.6B-GGUF",
filename="*Q4_0.gguf",
verbose=False,
logits_all=True,
)
allowed_tokens = ["Wrench", "Screwdriver", "Ornament"]
allowed_tokens_str = " | ".join(f'"{t}"' for t in allowed_tokens)
grammar = LlamaGrammar(
_grammar=f"""
root ::= allowed_tokens
start ::= ( allowed_tokens " " )*
allowed_tokens ::= {allowed_tokens_str}
"""
)
config = BeamSearchConfig(
max_depth=1,
end_token=None,
k_progress=[2, 3],
)
completion = BeamSearchCompletion(model, grammar, config)
message = "Hello?"
result = completion.completion_beam_search(message, max_tokens=5)
assert re.match(r"^((Wrench|Screwdriver|Ornament) ?)*$", result)
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
llama_cpp_beam_search-0.1.0.tar.gz
(140.1 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file llama_cpp_beam_search-0.1.0.tar.gz.
File metadata
- Download URL: llama_cpp_beam_search-0.1.0.tar.gz
- Upload date:
- Size: 140.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
7273af30f48df38fa015d41a27aa267e0027f97fdde6860e0755ce0f0625fd77
|
|
| MD5 |
650a0770e5786e5b9645e4eb194d8957
|
|
| BLAKE2b-256 |
3d74ddf24a250a5fecf13d6806c022e783377259137773f0357bf0ca7675a85d
|
Provenance
The following attestation bundles were made for llama_cpp_beam_search-0.1.0.tar.gz:
Publisher:
python-publish.yml on Dakantz/CHASTE
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
llama_cpp_beam_search-0.1.0.tar.gz -
Subject digest:
7273af30f48df38fa015d41a27aa267e0027f97fdde6860e0755ce0f0625fd77 - Sigstore transparency entry: 2722600650
- Sigstore integration time:
-
Permalink:
Dakantz/CHASTE@ed5aeade819aadb081953248d78a277c006266ef -
Branch / Tag:
refs/tags/beams-v0.1.0-re - Owner: https://github.com/Dakantz
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
python-publish.yml@ed5aeade819aadb081953248d78a277c006266ef -
Trigger Event:
release
-
Statement type:
File details
Details for the file llama_cpp_beam_search-0.1.0-py3-none-any.whl.
File metadata
- Download URL: llama_cpp_beam_search-0.1.0-py3-none-any.whl
- Upload date:
- Size: 159.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
c0867b529a92ec00fdfb0334a80ddc029141ad8b9c466b5f0516cd1702b71d78
|
|
| MD5 |
46dc2e6d337f2780f51a82cbf31d3d35
|
|
| BLAKE2b-256 |
b24ec2fcff566c3fadbfa858fa7fd81ece8decc985a3553894dc94b12b5a5473
|
Provenance
The following attestation bundles were made for llama_cpp_beam_search-0.1.0-py3-none-any.whl:
Publisher:
python-publish.yml on Dakantz/CHASTE
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
llama_cpp_beam_search-0.1.0-py3-none-any.whl -
Subject digest:
c0867b529a92ec00fdfb0334a80ddc029141ad8b9c466b5f0516cd1702b71d78 - Sigstore transparency entry: 2722601366
- Sigstore integration time:
-
Permalink:
Dakantz/CHASTE@ed5aeade819aadb081953248d78a277c006266ef -
Branch / Tag:
refs/tags/beams-v0.1.0-re - Owner: https://github.com/Dakantz
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
python-publish.yml@ed5aeade819aadb081953248d78a277c006266ef -
Trigger Event:
release
-
Statement type:
