🚀 Open World Agents
Everything you need to build state-of-the-art foundation multimodal desktop agent, end-to-end.
⚠️ Active Development Notice: This codebase is under active development. APIs and components may change, and some may be moved to separate repositories. Documentation may be incomplete or reference features still in development.
📄 Research Paper: This project was first introduced and developed for the D2E project. For more details, see D2E: Scaling Vision-Action Pretraining on Desktop Data for Transfer to Embodied AI. If you find this work useful, please cite our paper.
Quick Start
💡 This is a conceptual overview. See the Quick Start Guide for detailed instructions.
# 1. Record desktop interaction
$ ocap my-session.mcap
# 2. Process to training format
$ python scripts/01_raw_to_event.py --train-dir ./
# 3. Train your model (coming soon)
$ python train.py --dataset ./event-dataset
Installation
# For video recording, install GStreamer first. Skip if you only need data processing.
$ conda install open-world-agents::gstreamer-bundle
# Install OWA
$ pip install owa
Documentation
| Resource | Description |
|---|---|
| 🏠 Full Documentation | Complete docs with all guides and references |
| 📖 Quick Start Guide | Complete tutorial: Record → Process → Train |
| 🤗 Community Datasets | Browse and share datasets |
Core Components
- 🌍 Environment Framework: "USB-C of desktop agents" - universal interface for native desktop automation with pre-built plugins for desktop control, high-performance screen capture, and zero-configuration plugin system
- 📊 Data Infrastructure: Complete desktop agent data pipeline from recording to training with
OWAMcapformat - a universal standard powered by MCAP - 🛠️ CLI Tools: Command-line utilities (
owl) for recording, analyzing, and managing agent data - 🤖 Examples: Complete implementations and training pipelines for multimodal agents
Contributing
We welcome contributions! See our Contributing Guide.
License
MIT License. See LICENSE.
Citation
@article{choi2025d2e,
title={D2E: Scaling Vision-Action Pretraining on Desktop Data for Transfer to Embodied AI},
author={Choi, Suhwan and Jung, Jaeyoon and Seong, Haebin and Kim, Minchan and Kim, Minyeong and Cho, Yongjun and Kim, Yoonshik and Park, Yubeen and Yu, Youngjae and Lee, Yunsung},
journal={arXiv preprint arXiv:2510.05684},
year={2025}
}
Metadata
Release files for owa 0.6.6
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| owa-0.6.6.tar.gz | 17.8 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| owa-0.6.6-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 17.8 MB
Release files / owa-0.6.6.tar.gz
| Download URL | owa-0.6.6.tar.gz |
|---|---|
| Size | 17.8 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
6ae3eea9e4fb1adf556687886c96bf26d9abe20274c6086c8aaeaf5c622f9203
|
|
BLAKE2b-256 checksum How to use checksums |
0bd89fc99e3f9ad4b717bbfaa8c79dddcb0002d2d5c0e8be90c7e7fcbbd4a003
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.2 {"installer":{"name":"uv","version":"0.11.2","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":null,"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / owa-0.6.6-py3-none-any.whl
| Download URL | owa-0.6.6-py3-none-any.whl |
|---|---|
| Size | 3.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
2bf91caa5cec247115119db6466054640682102966cbbe14c51864b914bd2ff7
|
|
BLAKE2b-256 checksum How to use checksums |
3c2944a2424ca53997ffde479ecde13361a29524d3de0cd32b1de43c35f8a279
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.2 {"installer":{"name":"uv","version":"0.11.2","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":null,"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
