AutoRT - Pytorch
Project description
AutoRT
Implementation of AutoRT: "AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents". This repo will implement the multi agent system that transforms a scene into a list of ranked and priortized tasks for an robotic action model to execute. This is an very effective setup that I personally believe is the future for swarming robotic foundation models!
This project will be implemented using Swarms, for the various llms and use the official RT-1 as the robotic action model.
Install
$ pip3 install autort-swarms
Usage
AutoRTAgent
A single AutoRT agent that: analyzes a scene using visual COT -> generate tasks -> filter tasks -> execute it with a robotic transformer.
# Import necessary modules
import os
from autort import AutoRTSwarm, AutoRTAgent
# Set the OpenAI API key
openai_api_key = os.getenv("OPENAI_API_KEY")
# Define a list of AutoRTAgent instances
agents = [
AutoRTAgent(openai_api_key, max_tokens=1000),
AutoRTAgent(openai_api_key, max_tokens=1000),
]
# Create an instance of AutoRTSwarm with the agents and datastore
autort_swarm = AutoRTSwarm(agents)
# Run the AutoRTSwarm with the given inputs
autort_swarm.run(
"There is a bottle on the table.",
"https://i.imgur.com/2qY9f8U.png",
)
AutoRTSwarm
A team of AutoRT agents where you can plug in and play any number of AutoRTAgents
with customization. First, the task will be routed to each agent and then all of their outputs will be saved.
# Import necessary modules
import os
from autort import AutoRTSwarm, AutoRTAgent
# Set the OpenAI API key
openai_api_key = os.getenv("OPENAI_API_KEY")
# Define a list of AutoRTAgent instances
agents = [
AutoRTAgent(openai_api_key, max_tokens=1000),
AutoRTAgent(openai_api_key, max_tokens=1000),
]
# Create an instance of AutoRTSwarm with the agents and datastore
autort_swarm = AutoRTSwarm(agents)
# Run the AutoRTSwarm with the given inputs
autort_swarm.run(
"There is a bottle on the table.",
"https://i.imgur.com/2qY9f8U.png",
)
Citation
@inproceedings{
anonymous2023autort,
title={Auto{RT}: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents},
author={Anonymous},
booktitle={Submitted to The Twelfth International Conference on Learning Representations},
year={2023},
url={https://openreview.net/forum?id=xVlcbh0poD},
note={under review}
}
License
MIT
Todo
- Implement a run method into
AutoRTSwarm
that runs all the agents with APIs. - Make it able to send commands to a certain agent using the swarm network.
- Send a task to all agents in the swarm network
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file autort_swarms-0.0.6.tar.gz
.
File metadata
- Download URL: autort_swarms-0.0.6.tar.gz
- Upload date:
- Size: 9.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.3.2 CPython/3.11.0 Darwin/23.3.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | d35f1fbfe83d63a6bea378448e29d8488ee8fde15a72ca607cd737564767588a |
|
MD5 | 1238b668d040ba9b85d3b1b051e82352 |
|
BLAKE2b-256 | 6fe6dbe95e3e424a024f94a4e68fe1d11b036075d0ea9fa182ebd207fd9f9179 |
Provenance
File details
Details for the file autort_swarms-0.0.6-py3-none-any.whl
.
File metadata
- Download URL: autort_swarms-0.0.6-py3-none-any.whl
- Upload date:
- Size: 9.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.3.2 CPython/3.11.0 Darwin/23.3.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 73cc54b30201ab228008da1befa048b3b15d3b3f3c432374d7cf076c293e94c0 |
|
MD5 | 06bc0b75ca7a7459504effee2f52edca |
|
BLAKE2b-256 | 858a904d3b6ad1789b845d4b9e5b5fc4fbff79ce9156239c7b5dc0792091edae |