SENTIA
SENTIA is a PyTorch implementation of a text generation model combining multiple neural network architectures like GRUs, Transformers, MHAs and MEPA.
Installation
pip install sentia
Usage
import torch
from sentia import SENTIA
# Create model
model = SENTIA(vocab_size=10000, embedding_dim=512, num_heads=8, num_layers=6, hidden_dim=512)
# Forward pass
input_ids = torch.randint(0, 10000, (1,32))
outputs = model(input_ids)
# Generate text
generated = model.generate(input_ids, max_length=128)
Model Architecture
The SENTIA model consists of the following components:
- Embedding layer
- GRU layer
- MEPA (Mutation Enhanced Plasticity Architecture) layers
- Transformer decoder layers
- Multi-head attention layer
- Output head layers These components are combined to leverage the strengths of multiple architectures for improved text generation capabilities.
Training
The fit() method can bne used to train the model on a dataset. It handles the training loop, gradient accumulation, and RL calculations. Currently the scheduler parameter only supports StepLR
1 maintainer
Metadata
1 maintainer
Release files for sentia 1.17
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| sentia-1.17.tar.gz | 40.9 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| sentia-1.17-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 85.2 kB
Release files / sentia-1.17.tar.gz
| Download URL | sentia-1.17.tar.gz |
|---|---|
| Size | 40.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
9055b48d93e4ca80c926dd9aef4100e51b6944dedb8ae1abeeeb93bfe82a3f7a
|
|
BLAKE2b-256 checksum How to use checksums |
e831b958e0fc6067ae22e4c569d49d0ad8d9e57251b9ca8312b94b0532abaa41
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.11.6
|
Release files / sentia-1.17-py3-none-any.whl
| Download URL | sentia-1.17-py3-none-any.whl |
|---|---|
| Size | 44.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
8f1b97e35a4cf2f30bcd314b827dbdb40510c8ab64d00fa78a163a4546efeb2d
|
|
BLAKE2b-256 checksum How to use checksums |
25f50849c01280d703493ec2b2b01e60aa0b2eae9592c979f75dc0f341bf1755
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.11.6
|
