synthetic-transformers 0.2.4

transformers library new generate API

synthetic-transformers

Package structure

SyntheticTransformer

Class that will substitute the HuggingFace's PreTrainedModel API. The idea is to use one HuggingFace Transformer and a Tokenizer to initialize this class together with a prompt template and some special plugins as Hooks, Components and Commands. This will create an instance that then you can call as usual with the usual .generate() method.

The idea of this plugins is to give you more control over the generation and allow the LLM to interact with tools in a more dynamic manner.

Hooks

Hooks are one of the two main ways to change how your LLM generates text, they can be triggered by many events and will have access to needed information, such as generated text and others, depending on the activation. There are the following hook types:

• on-token: this hook runs when a token is generated by the model, it has access to the generated text and can modify it freely.
• on-eos: this hook runs when the end of generation exception has been raised.

Commands

Commands are executable functions that can be called during generation.