Microlib for sampling from an LLM
Project description
LLM Sampler
Install with:
pip install llm_sampler
llm_sampler allows you to sample from any LLM by given logits.
It is a collection of various sampling techniques found online.
For now, the methods are:
sample_huggingface- the one used in transformerssample_gpt_fast- the one used in gpt-fast
It also allows you get probability scores for sequences given by the user.
For example, if you supply the input:
Input: The sentiment of the sentence 'I loved it' is
- Option 0:
positive - Option 1:
negative - Option 2:
unknown
This lib will help you create_batch_with_continuations
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file llm_sampler-0.4.0.tar.gz.
File metadata
- Download URL: llm_sampler-0.4.0.tar.gz
- Upload date:
- Size: 5.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.11.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
dd3f789aa36b329e2bf9a52e77198dbd011f7628b8136e1362779a4ff1283931
|
|
| MD5 |
a754f9b60fd6e88fd439d8d4dfcc63a8
|
|
| BLAKE2b-256 |
623f45c5c2b5d7241dd5f84bf27dc2351bc6c05eee40f0ed87c1faafe531a64d
|
File details
Details for the file llm_sampler-0.4.0-py3-none-any.whl.
File metadata
- Download URL: llm_sampler-0.4.0-py3-none-any.whl
- Upload date:
- Size: 4.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.11.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
1b99d780695cfc1a58c807d3fc07d2f11726697ba06c25a8008d3972fc6c0342
|
|
| MD5 |
780ef79c4a8e22003aaf91fb819509a4
|
|
| BLAKE2b-256 |
efc961ebef8a144e38007e22c0c286983a90c2dab0953df184f9e6f85d0220ec
|
