AmadeusGPT🎻: We turn natural language descriptions of behaviors into machine-executable code
Project description
🪄 We turn natural language descriptions of behaviors into machine-executable code.
- We use large language models (LLMs) to bridge natural language and behavior analysis.
- This work is accepted to NeuRIPS2023! Read the paper, AmadeusGPT: a natural language interface for interactive animal behavioral analysis
- Like this project? Please consider giving us a star ⭐️!
Install & Run AmadeusGPT🎻
- AmadeusGPT is a Python package hosted on
pypi
. You can create a virtual env (conda, etc, see below*) or Docker and run:
pip install 'amadeusgpt[streamlit]'
- Please note that you need an openAI API key, which you can easily create here.
- If you want the Streamlit Demo on your computer, you will also need demo files that are supplied in our repo (see below**), so please git clone the repo and navigate into the
AmadeusGPT
directory. Then in your conda env/terminal runpip install '.[streamlit]'
. Then, to launch the Demo App execute in the terminal:
make app
Citation
If you use ideas or code from this project in your work, please cite us using the following BibTeX entry. 🙏
@article{ye2023amadeusGPT,
title={AmadeusGPT: a natural language interface for interactive animal behavioral analysis},
author={Shaokai Ye and Jessy Lauer and Mu Zhou and Alexander Mathis and Mackenzie Weygandt Mathis},
journal={Thirty-seventh Conference on Neural Information Processing Systems},
year={2023},
url={https://openreview.net/forum?id=9AcG3Tsyoq},
- arXiv preprint version AmadeusGPT: a natural language interface for interactive animal behavioral analysis by Shaokai Ye, Jessy Lauer, Mu Zhou, Alexander Mathis & Mackenzie W. Mathis.
Install tips
- *Make a new conda env:
conda create --name amadeusGPT python=3.9
then runconda activate amadeusGPT
or you can also use our supplied conda if you git cloned the repo (navigate into the conda directory):conda env create -f amadesuGPT.yml
then pip install amadeusGPT once created/launched. - **Git clone this repo: so please open a terminal, we recommend to download into Documents (so type
cd Documents
) and rungit clone https://github.com/AdaptiveMotorControlLab/AmadeusGPT.git
Then go into the dir (cd AmadeusGPT
) - If you want to use SAM, you need to download the weights. Otherwise you will see the following message in the app:
Cannot find SAM checkpoints. Skipping SAM
. Download them and add to "static" directory: wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth
Install trouble shooting:
- If you hit an error during installing on an M1/M2 Macbook with installing HDF5, run
conda install h5py
in your conda env. - If you launch the app and get an ffmpeg error,
RuntimeError: No ffmpeg exe could be found. Install ffmpeg on your system, or set the IMAGEIO_FFMPEG_EXE environment variable.
try runningconda install ffmpeg
. - If you have an M1/M2 chip and use CEBRA within AmadeusGPT, and you get this error:
RuntimeError: Device type MPS is not supported for torch.Generator() api
runpip install --upgrade torch
.
License
AmadeusGPT is license under the Apache-2.0 license.
- 🚨 Please note several key dependencies have their own licensing. Please carefully check the license information for DeepLabCut (LGPL-3.0 license), SAM (Apache-2.0 license), CEBRA (Non-Commercial), etc...
News
- 🤩 Dec 2023, code released!
- 🔥 Our work was accepted to NeuRIPS2023
- 🧙♀️ Open-source code coming in the fall of 2023
- 🔮 arXiv paper and demo released July 2023
- 🪄Contact us
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
amadeusgpt-0.0.2.tar.gz
(2.9 MB
view hashes)
Built Distribution
Close
Hashes for amadeusgpt-0.0.2-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a8c5307d6ccf4a57280f723fddb4297c1097671b36876a7405dd89ce3ee8283c |
|
MD5 | 5d550555ce26511399e4aa5803033935 |
|
BLAKE2b-256 | 08acf0171b714e677af7644c5e70437ef8cef20a15e9dece3e2918be18f25578 |