GeidiPrime
This is an extremely experimental Transformer architecture with Macaron like FFNs with local attention. Perhap's we can add the visual expert from Zeta and make it multi-modal!
Install
Usage
import torch
from geidi_prime.model import GeidiPrimeTransformer
model = GeidiPrimeTransformer(
dim=4096,
depth=6,
heads=8,
num_tokens=20000,
)
x = torch.randint(0, 20000, (1, 4096))
out = model(x)
print(out.shape)
License
MIT
1 maintainer
Metadata
1 maintainer
Release files for geidiprime 0.0.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| geidiprime-0.0.1.tar.gz | 3.9 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| geidiprime-0.0.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 7.7 kB
Release files / geidiprime-0.0.1.tar.gz
| Download URL | geidiprime-0.0.1.tar.gz |
|---|---|
| Size | 3.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
17627fb93375e8057383acd3495e15941e2d30c2550589b6a2b33b8ac026a98e
|
|
BLAKE2b-256 checksum How to use checksums |
2e7c8782659bc7836803167a41fd12964ba0610c4dc7ae2a2861b7e8a337dee8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
|
Release files / geidiprime-0.0.1-py3-none-any.whl
| Download URL | geidiprime-0.0.1-py3-none-any.whl |
|---|---|
| Size | 3.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
9a71e972687727f5c544e8fd7a88c48cba4fa69f297040d4e64043b1dc57c0df
|
|
BLAKE2b-256 checksum How to use checksums |
ff07c467733196302abf72c910fff56e5cf7927fbad9da34dd7aa432f31e7146
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
|
