Skip to main content

A Production Tool for Embodied AI.

Project description

:scroll:Loopquest

A Production Tool for Embodied AI. loopquest demo

Major features

  • Imitation Learning / Offline Reinforcement Learning Oriented MLOps. Log all the observation, action, reward, rendered images into database with only ONE extra line of code.
env = gymnasium.make("MountainCarContinuous-v0", render_mode="rgb_array")

->

import loopquest
loopquest.init()
env = loopquest.make_env(
    "MountainCarContinuous-v0", render_mode="rgb_array")
)
  • You can also evaluate the local policy or the policy saved in Huggingface directly by specifying number of episodes and number of steps for each episode.

Local Policy evaluation:

import loopquest
from loopquest.eval import evaluate_local_policy
from loopquest.policy.base import BasePolicy

class RandomPolicy(BasePolicy):
    def __init__(self, action_space):
        self.action_space = action_space

    def compute_action(self, observation):
        return self.action_space.sample()


policy = RandomPolicy(env.action_space)

evaluate_local_policy(
    policy,
    [
        "FetchPickAndPlace-v2",
        "FetchPushDense-v2",
        "FetchReachDense-v2",
        "FetchSlideDense-v2",
    ],
    num_episodes=1,
    num_steps_per_episode=100,
    project_name="test_robotics",
)

Remote Policy evaluation:

import loopquest
from loopquest.eval import evaluate_local_policy, evaluate_remote_policy

loopquest.init()

# Cloud evaluation example
evaluate_remote_policy(
    "jxx123/ppo-LunarLander-v2",
    "ppo-LunarLander-v2.zip",
    "PPO",
    ["LunarLander-v2"],
    num_episodes=1,
    num_steps_per_episode=100,
    project_name="test_lunar_remote",
    experiment_configs={"foo": [1, 2, 3], "bar": "hah", "bar": 1.1},
)
  • Directly trainable data for robotics foundation model. Select and download the (observation, action, reward) data with the dataloader interfaces of the most popular deep learning frameworks (e.g. tensorflow, pytorch, huggingface dataset apis). Check Dataset Quickstart Example for more details.
from loopquest.datasets import load_dataset, load_datasets
# Load data from a single experiment
ds = load_dataset("your_experiment_id")

# Load data from multiple experiments
ds = load_datasets(["exp1", "exp2"])

The data schema will look like

{
    'id': '34yixvic-0-1',
    'creation_time': '2023-09-03T20:53:30.603',
    'update_time': '2023-09-03T20:53:30.965',
    'experiment_id': '34yixvic',
    'episode': 0,
    'step': 1,
    'observation': [-0.55, 0.00],
    'action': [0.14],
    'reward': -0.00,
    'prev_observation': [-0.55, 0.00],
    'termnated': False,
    'truncated': False,
    'done': False,
    'info': '{}',
    'sub_goal': None,
    'image_ids': ['34yixvic-0-1-0'],
    'images': [<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=600x400 at 0x7F8D33094450>]
}
  • All the regular MLOps features are included, e.g. data visualization, simulation rendering, experiment management.

Installation

For stable version, run

pip install loopquest

For dev version or loopquest project contributors, clone the git to your local machine by running

git clone https://github.com/LoopMind-AI/loopquest.git

Change to the project root folder and install the package

cd loopquest
pip install -e .

Quick Start Examples

Run Local or Remote Eval

Run examples/run_local_eval.py.

import loopquest
from loopquest.eval import evaluate_local_policy
from loopquest.policy.base import BasePolicy
import gymnasium as gym


class RandomPolicy(BasePolicy):
    def __init__(self, action_space):
        self.action_space = action_space

    def compute_action(self, observation):
        return self.action_space.sample()


# Create this env just to get the action space.
env = gym.make("FetchPickAndPlace-v2")
policy = RandomPolicy(env.action_space)

loopquest.init()
evaluate_local_policy(
    policy,
    [
        "FetchPickAndPlace-v2",
        "FetchPushDense-v2",
        "FetchReachDense-v2",
        "FetchSlideDense-v2",
    ],
    num_episodes=1,
    num_steps_per_episode=100,
    project_name="test_robotics_new",
)

Run examples/run_remote_eval.py.

import loopquest
from loopquest.eval import evaluate_remote_policy


loopquest.init()
evaluate_remote_policy(
    "jxx123/ppo-LunarLander-v2",
    "ppo-LunarLander-v2.zip",
    "PPO",
    ["LunarLander-v2"],
    num_episodes=1,
    num_steps_per_episode=100,
    project_name="test_lunar_remote",
    experiment_configs={"foo": [1, 2, 3], "bar": "hah", "bar": 1.1},
)

Env Wrapper Example

Run examples/run_env_wrapper.py.

import loopquest

loopquest.init()
env = loopquest.make_env("MountainCarContinuous-v0", render_mode="rgb_array")
obs, info = env.reset()
for i in range(100):
    action = env.action_space.sample()
    obs, reward, terminated, truncated, info = env.step(action)
    rgb_array = env.render()
    if terminated or truncated:
        break
env.close()

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

loopquest-0.2.11.tar.gz (23.3 kB view details)

Uploaded Source

Built Distribution

loopquest-0.2.11-py3-none-any.whl (25.3 kB view details)

Uploaded Python 3

File details

Details for the file loopquest-0.2.11.tar.gz.

File metadata

  • Download URL: loopquest-0.2.11.tar.gz
  • Upload date:
  • Size: 23.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.11.8

File hashes

Hashes for loopquest-0.2.11.tar.gz
Algorithm Hash digest
SHA256 665a8cb87e2821223090f08125f37e13c39ca92ff766c8e9c530aad60cfc370f
MD5 15430ec6b2c77cf874ad1e0bfec0f9f0
BLAKE2b-256 69e0398ea66bbfc8d57a74043e566354b6c27d7ab2f75efe4a7e5361b675818e

See more details on using hashes here.

File details

Details for the file loopquest-0.2.11-py3-none-any.whl.

File metadata

  • Download URL: loopquest-0.2.11-py3-none-any.whl
  • Upload date:
  • Size: 25.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.11.8

File hashes

Hashes for loopquest-0.2.11-py3-none-any.whl
Algorithm Hash digest
SHA256 c18134e670791b1bcc2b9d23e7dc07976bc9616345b5460d5833999650b7be2c
MD5 d4a2779efe160a0c685f17084f28e18f
BLAKE2b-256 cba4a4fb6af5e754e5718221dd1b6b06da92fa17817ef7628fc4d71ed42af190

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page