Qwen-VL
My personal implementation of the model from "Qwen-VL: A Frontier Large Vision-Language Model with Versatile Abilities", they haven't released model code yet sooo... For more details, please refer to the full paper.
Install
pip3 install qwen
Usage
# Importing the necessary libraries
import torch
from qwen import Qwen
# Creating an instance of the Qwen model
model = Qwen()
# Generating random text and image tensors
text = torch.randint(0, 20000, (1, 1024))
img = torch.randn(1, 3, 256, 256)
# Passing the image and text tensors through the model
out = model(img, text) # (1, 1024, 20000)
Todo
-
Position aware vision language adapter, compresses image features. Singer layer cross attention module inited randomly => group of trainable embeddings as query vectors + image features from the visual encoder as keys for cross attention ops => OUTPUT: compresses visual feature sequence to a fixed lnegth of 256, 2d absolute positional encodings are integrated into the cross attentions mechanisms query key pairs => compressed feature sequence of length of 256 => fed into decoder llm
-
Bounding Boxes, for any given accurate bounding box, a norm process is applied in the range [0, 1000] and transformed into a string format (Xtope, Ytople)(Xottomright, Ybottomright) -> the string is tokenized as text and does not require positional vocabulary. Detection strings and regular text strings, two special tokens and are added to the beginning and end of the bounding box string. + another sed of special tokens ( and ) is introduced.
Citations
Please use the following to cite this work:
@article{bai2023qwen,
title={Qwen-VL: A Frontier Large Vision-Language Model with Versatile Abilities},
author={Bai, Jinze and Bai, Shuai and Yang, Shusheng and Wang, Shijie and Tan, Sinan and Wang, Peng and Lin, Junyang and Zhou, Chang and Zhou, Jingren},
journal={arXiv preprint arXiv:2308.12966},
year={2023},
url={https://doi.org/10.48550/arXiv.2308.12966}
}
Metadata
Release files for qwen 0.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| qwen-0.1.1.tar.gz | 4.4 kB | Without provenance | Details for qwen-0.1.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| qwen-0.1.1-py3-none-any.whl | Python 3 | none | any | Details for qwen-0.1.1-py3-none-any.whl |
Total release size: 8.7 kB
Release files / qwen-0.1.1.tar.gz
| Download URL | qwen-0.1.1.tar.gz |
|---|---|
| Size | 4.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
3aa2d2afd1c2842909f2e59ffce16a53fb6c02ba0993633d128dee17905c6afe
|
|
BLAKE2b-256 checksum How to use checksums |
55ec182ead9028328d988eb8f55b1da46d0e90789cfaa733e6cacae0d6c671dc
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
|
Release files / qwen-0.1.1-py3-none-any.whl
| Download URL | qwen-0.1.1-py3-none-any.whl |
|---|---|
| Size | 4.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
5c18e1e895195079ea7be7ee332c6eb2159a3dfddef2b47ef56daee5bd104d6c
|
|
BLAKE2b-256 checksum How to use checksums |
c2ad74d014e77c54a5221a67167184a233b18936cb9fb24ea58e0562ec781aea
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
|
Security for qwen 0.1.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.