amesa-inference-dev 0.32.0.dev38

Agent inference package using ONNX models without Ray or PyTorch dependencies

Amesa Inference

A lightweight inference package for running Amesa agents using ONNX models without Ray or PyTorch dependencies.

Overview

composabl_inference provides a standalone inference engine for running trained Amesa agents. It uses ONNX Runtime for model inference, making it suitable for deployment scenarios where you want to avoid heavy dependencies like Ray and PyTorch.

Features

• ONNX-based inference: Uses ONNX Runtime for efficient model inference
• No Ray or PyTorch dependencies: Lightweight package suitable for production deployment
• Network management: Supports both local and remote objects (agents, perceptors, controllers)
• Compatible API: Similar interface to Trainer.package() for easy migration

Installation

pip install amesa-inference

Usage

Basic Inference

from composabl_inference import InferenceEngine
from composabl_core import Agent

# Create inference engine (only license needed for license validation)
engine = InferenceEngine(license="your-license-key")

# Load agent
agent = Agent.load("path/to/agent")
await engine.load_agent(agent)

# Package agent for inference (similar to Trainer.package())
await engine.package()

# Run inference
observation = {...}  # Your observation from the simulator
action = engine.execute(observation)

With Remote Objects

The inference engine supports remote agents, perceptors, and controllers, just like the Trainer:

from composabl_inference import InferenceEngine

# Optional: provide custom config for NetworkMgr (e.g., for remote targets)
config = {
    "target": {
        "local": {
            "address": "localhost:1337",
        },
    },
}

engine = InferenceEngine(license="your-license-key", config=config)
await engine.load_agent("path/to/agent")
await engine.package()

# The agent processor will automatically handle remote objects
action = engine.execute(observation)

Cleanup

# Clean up resources
await engine.close()

Architecture

Components

1. InferenceEngine: Main entry point for inference operations
2. NetworkMgr: Manages network connections (non-Ray version)
3. ONNXInferenceEngine: Handles ONNX model loading and inference
4. ONNXAgentProcessor: Processes agents using ONNX models instead of PyTorch

Differences from Trainer

• Uses ONNX Runtime instead of PyTorch for model inference
• NetworkMgr is not a Ray actor (runs in the same process)
• No Ray initialization required
• Lighter weight, suitable for production deployment

Requirements

• Python >= 3.10
• composabl-core
• composabl-api
• onnxruntime
• numpy

License

Proprietary and confidential - Copyright (C) Amesa, Inc