GeidiPrime
This is an extremely experimental Transformer architecture with Macaron like FFNs with local attention. Perhap's we can add the visual expert from Zeta and make it multi-modal!
Install
Usage
import torch
from geidi_prime.model import GeidiPrimeTransformer
model = GeidiPrimeTransformer(
dim=4096,
depth=6,
heads=8,
num_tokens=20000,
)
x = torch.randint(0, 20000, (1, 4096))
out = model(x)
print(out.shape)
License
MIT
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
geidiprime-0.0.1.tar.gz
(3.9 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file geidiprime-0.0.1.tar.gz.
File metadata
- Download URL: geidiprime-0.0.1.tar.gz
- Upload date:
- Size: 3.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
17627fb93375e8057383acd3495e15941e2d30c2550589b6a2b33b8ac026a98e
|
|
| MD5 |
8088a1df2b17c51f26e08ca6c81216c0
|
|
| BLAKE2b-256 |
2e7c8782659bc7836803167a41fd12964ba0610c4dc7ae2a2861b7e8a337dee8
|
File details
Details for the file geidiprime-0.0.1-py3-none-any.whl.
File metadata
- Download URL: geidiprime-0.0.1-py3-none-any.whl
- Upload date:
- Size: 3.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
9a71e972687727f5c544e8fd7a88c48cba4fa69f297040d4e64043b1dc57c0df
|
|
| MD5 |
c1ce47a00a657655b6c22ab718a920ff
|
|
| BLAKE2b-256 |
ff07c467733196302abf72c910fff56e5cf7927fbad9da34dd7aa432f31e7146
|
