RAI
RAI is a flexible AI agent framework to develop and deploy Embodied AI features for your robots.
Installation
- Install RAI:
pip install rai-core
- Initialize the global configuration file:
rai-config-init
- Optionally install ROS 2 dependencies:
sudo apt install ros-${ROS_DISTRO}-rai-interfaces
For more visit robotecai.github.io/rai for the latest documentation, setup guide and tutorials.
🎯 Overview
| Category | Description | Features |
|---|---|---|
| 🤖 Multi-Agent Systems | Empowering robotics with advanced AI capabilities | • Seamlessly integrate Gen AI capabilities into your robots • Enable sophisticated agent-based architectures |
| 🔄 Robot Intelligence | Enhancing robotic systems with smart features | • Add natural human-robot interaction capabilities • Bring flexible problem-solving to your existing stack • Provide ready-to-use AI features out of the box |
| 🌟 Multi-Modal Interaction | Supporting diverse interaction capabilities | • Handle diverse data types natively • Enable rich sensory integration • Process multiple input/output modalities simultaneously |
RAI framework
- rai core: Core functionality for multi-agent system, human-robot interaction and multi-modalities.
- rai whoami: Tool to extract and synthesize robot embodiment information from a structured directory of documentation, images, and URDFs.
- rai_asr: Speech-to-text models and tools.
- rai_tts: Text-to-speech models and tools.
- rai_sim: Package for connecting RAI to simulation environments.
- rai_bench: Benchmarking suite for RAI. Test agents, models, tools, simulators, etc.
- rai_perception: Perception module for open-set detection models and tools.
- rai_nomad: Integration with NoMaD for navigation.
- rai_finetune: Finetune LLMs on your embodied data.
Simulation demos
Try RAI yourself with these demos:
| Application | Robot | Description | Docs Link |
|---|---|---|---|
| Mission and obstacle reasoning in orchards | Autonomous tractor | In a beautiful scene of a virtual orchard, RAI goes beyond obstacle detection to analyze best course of action for a given unexpected situation. | link |
| Manipulation tasks with natural language | Robot Arm (Franka Panda) | Complete flexible manipulation tasks thanks to RAI and Grounded SAM 2 | link |
| Autonomous mobile robot demo | Husarion ROSbot XL | Demonstrate RAI's interaction with an autonomous mobile robot platform for navigation and control | link |
| Speech-to-speech interaction with autonomous taxi | Simulated car | Demonstrate RAI's speech-to-speech interaction capabilities for specifying destinations to an autonomous taxi in awsim with autoware environment | link |
Community
Embodied AI Community Group
RAI is one of the main projects in focus of the Embodied AI Community Group. If you would like to join the next meeting, look for it in the ROS Community Calendar.
Publicity
- A talk about RAI at ROSCon 2024.
RAI Q&A
Please take a look at Q&A.
Developer Resources
See our documentation for a deeper dive into RAI, including instructions on creating a configuration specifically for your robot.
Contributing
You are welcome to contribute to RAI! Please see our Contribution Guide.
4 maintainers
Metadata
4 maintainers
Release files for rai_core 2.12.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| rai_core-2.12.1.tar.gz | 82.3 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| rai_core-2.12.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 227.0 kB
Release files / rai_core-2.12.1.tar.gz
| Download URL | rai_core-2.12.1.tar.gz |
|---|---|
| Size | 82.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
dc27059fa1c088cdfdac0d48a66cc57931812861be2b7aeca804db07ea6317ca
|
|
BLAKE2b-256 checksum How to use checksums |
c4387691b1c01f7eef4f8e5e374fa261b99bc6f50889f6b830795d3cbcb27835
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.19 {"installer":{"name":"uv","version":"0.11.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / rai_core-2.12.1-py3-none-any.whl
| Download URL | rai_core-2.12.1-py3-none-any.whl |
|---|---|
| Size | 144.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
e38b90691710d1d2ddb00feabef8d5366d54c2de85cff32429fe73aecd8a05ab
|
|
BLAKE2b-256 checksum How to use checksums |
66f231694fd7bd4af9edbb70bc25e875554464b4937904cf9e37efcca3407c82
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.19 {"installer":{"name":"uv","version":"0.11.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
