llm-sampler 0.4.0

Microlib for sampling from an LLM

LLM Sampler

Install with:

pip install llm_sampler

llm_sampler allows you to sample from any LLM by given logits.

It is a collection of various sampling techniques found online.

For now, the methods are:

1. sample_huggingface - the one used in transformers
2. sample_gpt_fast - the one used in gpt-fast

It also allows you get probability scores for sequences given by the user.

For example, if you supply the input: Input: The sentiment of the sentence 'I loved it' is

• Option 0: positive
• Option 1: negative
• Option 2: unknown

This lib will help you create_batch_with_continuations