NVIDIA Plugin for Vision Agents
NVIDIA VLM integration for Vision Agents. Supports vision language models through NVIDIA's Chat Completions API with NVCF asset management.
Features
- Video understanding: Automatically buffers and forwards video frames to NVIDIA VLM models
- Streaming responses: Supports streaming text responses with real-time chunk events
- Asset management: Automatically uploads frames as assets and cleans them up after use
- Configurable frame rate and buffer duration for optimal performance
Installation
uv add "vision-agents[nvidia]"
# or directly
uv add vision-agents-plugins-nvidia
Configuration
Set your NVIDIA API key:
export NVIDIA_API_KEY=your_nvidia_api_key
Usage
Basic VLM Usage
from vision_agents.plugins import nvidia
vlm = nvidia.VLM(
model="nvidia/cosmos-reason2-8b",
fps=1,
frame_buffer_seconds=10,
)
# VLM automatically buffers video frames when used with an Agent
response = await vlm.simple_response("What do you see?")
print(response.text)
With Custom Configuration
from vision_agents.plugins import nvidia
vlm = nvidia.VLM(
model="nvidia/cosmos-reason2-8b",
api_key="your-api-key",
fps=2,
frame_buffer_seconds=15,
frame_width=1280,
frame_height=720,
max_tokens=2048,
temperature=0.3,
top_p=0.8,
)
Configuration
| Parameter | Description | Default | Type |
|---|---|---|---|
model |
NVIDIA model ID | "nvidia/cosmos-reason2-8b" |
str |
api_key |
NVIDIA API token (or use NVIDIA_API_KEY env var) |
None |
Optional[str] |
fps |
Frames per second to buffer | 1 |
int |
frame_buffer_seconds |
Number of seconds to buffer | 10 |
int |
frame_width |
Width of video frames to send | 800 |
int |
frame_height |
Height of video frames to send | 600 |
int |
max_tokens |
Maximum response tokens | 1024 |
int |
temperature |
Temperature for sampling | 0.2 |
float |
top_p |
Top-p sampling parameter | 0.7 |
float |
frames_per_second |
Frames per second for video models | 8 |
int |
Dependencies
vision-agents: Core Vision Agents frameworkaiohttp>=3.9.0: Async HTTP client for API requests
2 maintainers
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file vision_agents_plugins_nvidia-0.6.7.tar.gz.
File metadata
- Download URL: vision_agents_plugins_nvidia-0.6.7.tar.gz
- Upload date:
- Size: 7.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
uv/0.10.10 {"installer":{"name":"uv","version":"0.10.10","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
4544111cb93db6760b5359c2fa83ff300323e93d4a0e6cf525e0ee554c2f96d6
|
|
| MD5 |
3dbbadbc96b87099b7a2be3ecfb81da2
|
|
| BLAKE2b-256 |
318f0829e6ee71ac56da3002b5f9c5356fa696fe7439deda2eeb4e92e962f4bf
|
File details
Details for the file vision_agents_plugins_nvidia-0.6.7-py3-none-any.whl.
File metadata
- Download URL: vision_agents_plugins_nvidia-0.6.7-py3-none-any.whl
- Upload date:
- Size: 7.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
uv/0.10.10 {"installer":{"name":"uv","version":"0.10.10","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
462e5a4d510430f0b1ce72341c4d7d56874f625f1875bc318938faacb5dfaa46
|
|
| MD5 |
b54f88ed06610d1970a3b68251b86a5f
|
|
| BLAKE2b-256 |
4468f50560381ba9847e6e2e0fe4ef2219b18e2f2a0e7ee7ff6e3c19f69249bd
|
