Paper - Pytorch
Project description
GeidiPrime
This is an extremely experimental Transformer architecture with Macaron like FFNs with local attention. Perhap's we can add the visual expert from Zeta and make it multi-modal!
Install
Usage
import torch
from geidi_prime.model import GeidiPrimeTransformer
model = GeidiPrimeTransformer(
dim=4096,
depth=6,
heads=8,
num_tokens=20000,
)
x = torch.randint(0, 20000, (1, 4096))
out = model(x)
print(out.shape)
License
MIT
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
geidiprime-0.0.1.tar.gz
(3.9 kB
view details)
Built Distribution
File details
Details for the file geidiprime-0.0.1.tar.gz.
File metadata
- Download URL: geidiprime-0.0.1.tar.gz
- Upload date:
- Size: 3.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 17627fb93375e8057383acd3495e15941e2d30c2550589b6a2b33b8ac026a98e |
|
| MD5 | 8088a1df2b17c51f26e08ca6c81216c0 |
|
| BLAKE2b-256 | 2e7c8782659bc7836803167a41fd12964ba0610c4dc7ae2a2861b7e8a337dee8 |
File details
Details for the file geidiprime-0.0.1-py3-none-any.whl.
File metadata
- Download URL: geidiprime-0.0.1-py3-none-any.whl
- Upload date:
- Size: 3.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 9a71e972687727f5c544e8fd7a88c48cba4fa69f297040d4e64043b1dc57c0df |
|
| MD5 | c1ce47a00a657655b6c22ab718a920ff |
|
| BLAKE2b-256 | ff07c467733196302abf72c910fff56e5cf7927fbad9da34dd7aa432f31e7146 |
