Microlib for sampling from an LLM
Project description
LLM Sampler
Install with:
pip install llm_sampler
llm_sampler allows you to sample from any LLM by given logits.
It is a collection of various sampling techniques found online.
For now, the methods are:
sample_huggingface- the one used in transformerssample_gpt_fast- the one used in gpt-fast
It also allows you get probability scores for sequences given by the user.
For example, if you supply the input:
Input: The sentiment of the sentence 'I loved it' is
- Option 0:
positive - Option 1:
negative - Option 2:
unknown
This lib will help you create_batch_with_continuations
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
llm_sampler-0.4.0.tar.gz
(5.0 kB
view details)
Built Distribution
File details
Details for the file llm_sampler-0.4.0.tar.gz.
File metadata
- Download URL: llm_sampler-0.4.0.tar.gz
- Upload date:
- Size: 5.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.11.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | dd3f789aa36b329e2bf9a52e77198dbd011f7628b8136e1362779a4ff1283931 |
|
| MD5 | a754f9b60fd6e88fd439d8d4dfcc63a8 |
|
| BLAKE2b-256 | 623f45c5c2b5d7241dd5f84bf27dc2351bc6c05eee40f0ed87c1faafe531a64d |
File details
Details for the file llm_sampler-0.4.0-py3-none-any.whl.
File metadata
- Download URL: llm_sampler-0.4.0-py3-none-any.whl
- Upload date:
- Size: 4.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.11.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 1b99d780695cfc1a58c807d3fc07d2f11726697ba06c25a8008d3972fc6c0342 |
|
| MD5 | 780ef79c4a8e22003aaf91fb819509a4 |
|
| BLAKE2b-256 | efc961ebef8a144e38007e22c0c286983a90c2dab0953df184f9e6f85d0220ec |
