oneinfer-cli 0.3.4

Seamlessly deploy local Docker containers to OneInfer GPU instances over SSH.

OneInfer Developer CLI 🚀

The official, open-source Developer CLI for seamlessly testing and deploying local Docker containers to OneInfer GPU instances over SSH.

⚠️ Note: OneInfer GPU instances are secure, unprivileged containers. They do not run a full dockerd daemon. Therefore, you cannot simply docker run inside them. This CLI handles the complexity of tunneling your image and running it via user-space container runtimes (udocker) automatically!

Installation

pip install oneinfer-cli

(Optionally use pipx install oneinfer-cli for global isolation).

Getting Started

1. Login: Authenticate your CLI with your OneInfer Account.

   oneinfer login
   # Paste your API Token when prompted
   

2. Build Your AI App: Build your standard Docker container locally.

   docker build -t my-custom-llm:v1 .
   

3. Deploy to a GPU Instance: Push and run it directly to your running OneInfer instance.

   oneinfer deploy my-custom-llm:v1 --instance-id xyz-1234
   

What happens under the hood?

When you run deploy, the CLI:

1. Talks to the OneInfer API to securely fetch your instance's SSH dynamically.
2. Runs docker save locally to bundle your image.
3. Transmits the bundle over SSH via SCP.
4. Instructs the remote instance to load and run the container securely in user-space with full GPU passthrough using udocker --allow-root run --nv my-custom-llm:v1.

Contributing

We welcome PRs! Ensure you have hatch installed, and run pip install -e . to develop locally.