This release is a pre-release and may not be stable for production use.
Visual ChatGPT
Visual ChatGPT connects ChatGPT and a series of Visual Foundation Models to enable sending and receiving images during chatting.
See our paper: Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models
Demo
System Architecture
Quick Start
# create a new environment
conda create -n visgpt python=3.8
# activate the new environment
conda activate visgpt
# prepare the basic environments
pip install -r requirement.txt
# download the visual foundation models
bash download.sh
# prepare your private openAI private key
export OPENAI_API_KEY={Your_Private_Openai_Key}
# create a folder to save images
mkdir ./image
# Start Visual ChatGPT !
python visual_chatgpt.py
GPU memory usage
Here we list the GPU memory usage of each visual foundation model, one can modify self.tools with fewer visual foundation models to save your GPU memory:
| Foundation Model | Memory Usage (MB) |
|---|---|
| ImageEditing | 6667 |
| ImageCaption | 1755 |
| T2I | 6677 |
| canny2image | 5540 |
| line2image | 6679 |
| hed2image | 6679 |
| scribble2image | 6679 |
| pose2image | 6681 |
| BLIPVQA | 2709 |
| seg2image | 5540 |
| depth2image | 6677 |
| normal2image | 3974 |
| InstructPix2Pix | 2795 |
Acknowledgement
We appreciate the open source of the following projects:
Hugging Face LangChain Stable Diffusion ControlNet InstructPix2Pix CLIPSeg BLIP
Metadata
Release files for visualchatgpt 0.0.1.dev0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| visualchatgpt-0.0.1.dev0.linux-x86_64.tar.gz | 2.0 kB | Without provenance | Details for visualchatgpt-0.0.1.dev0.linux-x86_64.tar.gz |
Release files / visualchatgpt-0.0.1.dev0.linux-x86_64.tar.gz
| Download URL | visualchatgpt-0.0.1.dev0.linux-x86_64.tar.gz |
|---|---|
| Size | 2.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
24c1bec087a1f8d1d2807d75782fb4f5a955e3dcf147caebf10d5712107a1a38
|
|
BLAKE2b-256 checksum How to use checksums |
38457a59ee00311e6222ce94f4ea148497cf73abaee206ef6bd4d97936b4c634
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.8.14
|
Security for visualchatgpt 0.0.1.dev0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.