Attention-and-Transformers

Attention mechanisms and Transformers

This goal of this repository is to host basic architecture and model traning code associated with the different attention mechanisms and transformer architecture.
At the moment, I more interested in learning and recreating these new architectures from scratch than full-fledged training. For now, I'll just be training these models on small datasets.

Installation

Using pip to install from pypi

pip install Attention-and-Transformers

Using pip to install latest version from github

pip install git+https://github.com/veb-101/Attention-and-Transformers.git

Local clone and install

git clone https://github.com/veb-101/Attention-and-Transformers.git atf
cd atf
python setup.py install

Example Use

python load_test.py

Attention Mechanisms

# No.	Mechanism	Paper
1	Multi-head Self Attention	Attention is all you need
2	Multi-head Self Attention 2D	MobileViT V1
2	Separable Self Attention	MobileViT V2

Transformer Models

# No.	Models	Paper
1	Vision Transformer	An Image is Worth 16x16 Words:
2	MobileViT-V1	MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer
3	MobileViT-V2	Separable Self-attention for Mobile Vision Transformers

Metadata

Release files for Attention-and-Transformers 0.0.15

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for Attention-and-Transformers 0.0.15
File	Size	Uploaded	Provenance
Attention_and_Transformers-0.0.15.tar.gz	17.5 kB	Dec 17, 2022	Without provenance	Details

Built distribution (wheel)

Table of built distributions (wheels) for Attention-and-Transformers 0.0.15
File	Interpreter	ABI	Platform	Reset
Attention_and_Transformers-0.0.15-py3-none-any.whl 24.8 kB Dec 17, 2022 Without provenance	Python 3	none	any	Details

Total release size: 42.3 kB

Release files / Attention_and_Transformers-0.0.15.tar.gz

Download URL	Attention_and_Transformers-0.0.15.tar.gz
Size	17.5 kB
Tags	Source
SHA-256 checksum How to use checksums	`18de0593625a77b0dacff19e64ef77b6860e4e9a8d6f06e99f9448c127f0fd07`
BLAKE2b-256 checksum How to use checksums	`0c5d1b143ff86a9182751ac0ddabac126afce2f1c0e6ee91e812f46d0c9d2b3f`
Upload date	Dec 17, 2022
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/4.0.1 CPython/3.10.4`

Release files / Attention_and_Transformers-0.0.15-py3-none-any.whl

Download URL	Attention_and_Transformers-0.0.15-py3-none-any.whl
Size	24.8 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`a32c67a0fcb200627baad4f66e7bcec4edc96771f1faf67d7af1c669ce139ae3`
BLAKE2b-256 checksum How to use checksums	`862c83acacb0fa37c7e47809d896287e2440ba66682f4f948e423148dcca8482`
Upload date	Dec 17, 2022
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/4.0.1 CPython/3.10.4`

Security for Attention-and-Transformers 0.0.15

Report project as malware

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.