Skip to main content

AutoRT - Pytorch

Project description

Multi-Modality

AutoRT

AutoRTImage Implementation of AutoRT: "AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents". This repo will implement the multi agent system that transforms a scene into a list of ranked and priortized tasks for an robotic action model to execute. This is an very effective setup that I personally believe is the future for swarming robotic foundation models!

This project will be implemented using Swarms, for the various llms and use the official RT-1 as the robotic action model.

PAPER LINK

Install

$ pip3 install autort-swarms

Usage

AutoRTAgent

A single AutoRT agent that: analyzes a scene using visual COT -> generate tasks -> filter tasks -> execute it with a robotic transformer.

# Import necessary modules
import os
from autort import AutoRTSwarm, AutoRTAgent

# Set the OpenAI API key
openai_api_key = os.getenv("OPENAI_API_KEY")

# Define a list of AutoRTAgent instances
agents = [
    AutoRTAgent(openai_api_key, max_tokens=1000),
    AutoRTAgent(openai_api_key, max_tokens=1000),
]

# Create an instance of AutoRTSwarm with the agents and datastore
autort_swarm = AutoRTSwarm(agents)

# Run the AutoRTSwarm with the given inputs
autort_swarm.run(
    "There is a bottle on the table.",
    "https://i.imgur.com/2qY9f8U.png",
)

AutoRTSwarm

A team of AutoRT agents where you can plug in and play any number of AutoRTAgents with customization. First, the task will be routed to each agent and then all of their outputs will be saved.

# Import necessary modules
import os
from autort import AutoRTSwarm, AutoRTAgent

# Set the OpenAI API key
openai_api_key = os.getenv("OPENAI_API_KEY")

# Define a list of AutoRTAgent instances
agents = [
    AutoRTAgent(openai_api_key, max_tokens=1000),
    AutoRTAgent(openai_api_key, max_tokens=1000),
]

# Create an instance of AutoRTSwarm with the agents and datastore
autort_swarm = AutoRTSwarm(agents)

# Run the AutoRTSwarm with the given inputs
autort_swarm.run(
    "There is a bottle on the table.",
    "https://i.imgur.com/2qY9f8U.png",
)

Citation

@inproceedings{
    anonymous2023autort,
    title={Auto{RT}: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents},
    author={Anonymous},
    booktitle={Submitted to The Twelfth International Conference on Learning Representations},
    year={2023},
    url={https://openreview.net/forum?id=xVlcbh0poD},
    note={under review}
}

License

MIT

Todo

  • Implement a run method into AutoRTSwarm that runs all the agents with APIs.
  • Make it able to send commands to a certain agent using the swarm network.
  • Send a task to all agents in the swarm network

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

autort_swarms-0.0.6.tar.gz (9.3 kB view details)

Uploaded Source

Built Distribution

autort_swarms-0.0.6-py3-none-any.whl (9.3 kB view details)

Uploaded Python 3

File details

Details for the file autort_swarms-0.0.6.tar.gz.

File metadata

  • Download URL: autort_swarms-0.0.6.tar.gz
  • Upload date:
  • Size: 9.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.3.2 CPython/3.11.0 Darwin/23.3.0

File hashes

Hashes for autort_swarms-0.0.6.tar.gz
Algorithm Hash digest
SHA256 d35f1fbfe83d63a6bea378448e29d8488ee8fde15a72ca607cd737564767588a
MD5 1238b668d040ba9b85d3b1b051e82352
BLAKE2b-256 6fe6dbe95e3e424a024f94a4e68fe1d11b036075d0ea9fa182ebd207fd9f9179

See more details on using hashes here.

Provenance

File details

Details for the file autort_swarms-0.0.6-py3-none-any.whl.

File metadata

  • Download URL: autort_swarms-0.0.6-py3-none-any.whl
  • Upload date:
  • Size: 9.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.3.2 CPython/3.11.0 Darwin/23.3.0

File hashes

Hashes for autort_swarms-0.0.6-py3-none-any.whl
Algorithm Hash digest
SHA256 73cc54b30201ab228008da1befa048b3b15d3b3f3c432374d7cf076c293e94c0
MD5 06bc0b75ca7a7459504effee2f52edca
BLAKE2b-256 858a904d3b6ad1789b845d4b9e5b5fc4fbff79ce9156239c7b5dc0792091edae

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page