palme - Pytorch
Project description
🌴 PALM-E: A Multi-Modal AI Model
This is the open source implementation of the SOTA multi-modality foundation model "PaLM-E: An Embodied Multimodal Language Model" from Google, PALM-E is a single large embodied multimodal model, that can address a variety of embodied reasoning tasks, from a variety of observation modalities, on multiple embodiments, and further, exhibits positive transfer: the model benefits from diverse joint training across internet-scale language, vision, and visual-language domains.
🚀 Quick Start
Installation 📦
pip install palme
Usage 🎨
import torch
from palme import PalmE
#usage
img = torch.randn(1, 3, 256, 256)
caption_tokens = torch.randint(0, 4)
model = PalmE()
output = model(img, caption_tokens)
Contribute || Be Part of the PALM-E Adventure 🤝
Your brilliance is needed! Join us, and together, let's make PALM-E even more awe-inspiring:
- Get Your Copy: Fork the PALM-E repo.
- Make It Local: Clone your fork.
- Prep Your Tools: Install the necessities.
- Discover & Innovate: Dive into the code.
- Craft Your Magic: Branch and code away.
- Show & Tell: Push your changes and craft a pull request.
🐞 Fixes, 🎨 enhancements, 📝 docs, or 💡 ideas – all are welcome! Let's shape the future of AI, hand in hand.
Roadmap
- 🕵️ Verify decoder configurations.
- 🚂 Recreate the training strategy detailed in the paper.
- 🌐 Train on the datasets used in the paper.
📘 Documentation
- Documentation will come soon
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.