synthetic-transformers 0.1.8

transformers library new generate API

synthetic-transformers

Package structure

SyntheticTransformer

Class that will substitute the HuggingFace's PreTrainedModel API. The idea is to use one HuggingFace Transformer and a Tokenizer to initialize this class together with a prompt template and some special plugins as Hooks, Components and Commands. This will create an instance that then you can call as usual with the usual .generate() method.

The idea of this plugins is to give you more control over the generation and allow the LLM to interact with tools in a more dynamic manner.

Hooks

Hooks are one of the two main ways to change how your LLM generates text, they can be triggered by many events and will have access to needed information, such as generated text and others, depending on the activation. There are the following hook types:

• on-token: this hook runs when a token is generated by the model, it has access to the generated text and can modify it freely.
• on-command:{command-id}: this hook runs after a command has finished running, it has access to the command input and output and to the
• on-eos: this hook runs when the end of generation exception has been raised.
• on-change:{component-id}: this hook runs when a component reports a change.

Commands

Commands are the other way to change how your LLM generates text, they are called by the CommandCall hook that is always added as the last hook in the on-token type. This commands have read and write access to the content and the input of the command.

TODO: Finish this section