Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

Get Started with Microsoft Agent Framework NVIDIA NIM

An alpha NVIDIA NIM embedding integration for Microsoft Agent Framework. Please install this package:

pip install agent-framework-nvidia --pre

and see the README for more information.

See the NVIDIA samples for a runnable example.

Embedding Client

The NvidiaEmbeddingClient provides embedding generation using models hosted on NVIDIA NIM. NIM exposes an OpenAI-compatible surface, so this client builds on the OpenAI embedding client rather than reimplementing the wire format.

Quick Start

from agent_framework_nvidia import NvidiaEmbeddingClient

# Using environment variables (NVIDIA_API_KEY, NVIDIA_EMBEDDING_MODEL)
# Parameters can also be passed directly:
# NvidiaEmbeddingClient(model="nvidia/nemotron-3-embed-1b", api_key="your-api-key")
client = NvidiaEmbeddingClient()

result = await client.get_embeddings(["Hello, world!", "How are you?"])
for embedding in result:
    print(f"Dimensions: {embedding.dimensions}")
    print(f"Vector: {embedding.vector[:5]}...")

# Close the HTTP client the embedding client created. A client passed as async_client is left open.
await client.close()

Configuration

Environment Variable Description
NVIDIA_API_KEY Your NVIDIA API key, from build.nvidia.com
NVIDIA_EMBEDDING_MODEL Embedding model name (e.g., nvidia/nemotron-3-embed-1b)
NVIDIA_BASE_URL Optional endpoint override, for a self-hosted NIM container

The service throttles bursts with 429, which the OpenAI SDK retries twice by default. For heavy parallel ingestion, pass a pre-configured async_client, for example AsyncOpenAI(..., max_retries=5), to change retries or timeouts.

Options

NvidiaEmbeddingOptions adds two NVIDIA fields to the OpenAI embedding options:

Option Values Purpose
input_type "passage", "query" Asymmetric retrieval models embed documents and search text differently
truncate "NONE", "START", "END" Reject over-long input, or drop tokens from that end

For retrieval, embed stored documents as passages and search text as queries. Vector collections forward embeddings_options to each request:

await collection.upsert(notes, embeddings_options={"input_type": "passage", "truncate": "END"})
results = await collection.search("How do I call a model?", embeddings_options={"input_type": "query"})

create_vector_search_tool(..., embeddings_options={"input_type": "query"}) does the same for the tool an agent calls. nvidia/nemotron-3-embed-1b accepts only 2048 dimensions, so declare that on the vector field.

The hosted endpoint accepts at most 256 inputs and a 2 MiB body per request, so the client sends larger calls, such as an upsert of many records, as consecutive requests within both limits and returns the vectors in order.

Without truncate, the service shortens input over the model's token limit itself, but it rejects any input over 65,536 characters, and one such document fails the whole call, so nothing in that upsert is written. Set truncate="END" (or "START") when ingesting documents of unknown length. truncate="NONE" rejects input over the token limit instead.

Choosing a model

Model availability is per-account: the endpoint lists more embedding models than a given API key is entitled to call, and an unentitled model returns 404 ... Not found for account rather than a permission error. Retired models return 410 Gone. If a model name from the documentation does not work, list the models available to your key rather than assuming the name is wrong. An invalid key returns 403 Forbidden with Authorization failed, not 401.

Metadata

Release files for agent-framework-nvidia 1.0.0a261030

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for agent-framework-nvidia 1.0.0a261030
File Size Uploaded Provenance Actions
agent_framework_nvidia-1.0.0a261030.tar.gz 7.6 kB Without provenance Details for agent_framework_nvidia-1.0.0a261030.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for agent-framework-nvidia 1.0.0a261030
File Interpreter ABI Platform Actions
agent_framework_nvidia-1.0.0a261030-py3-none-any.whl Python 3 none any Details for agent_framework_nvidia-1.0.0a261030-py3-none-any.whl

Total release size: 16.5 kB

Release files / agent_framework_nvidia-1.0.0a261030.tar.gz

Download URL agent_framework_nvidia-1.0.0a261030.tar.gz
Size 7.6 kB
Tags Source
SHA-256 checksum
How to use checksums
16c7867f5c190e314594928dd92a585ab2cfb6aa72483ab05ad08e37fb430f81
BLAKE2b-256 checksum
How to use checksums
846be35616bfdbbaad2fa5776013136141bc1629f4f346acb1cb97d23a6e81af
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via RestSharp/106.13.0.0

Release files / agent_framework_nvidia-1.0.0a261030-py3-none-any.whl

Download URL agent_framework_nvidia-1.0.0a261030-py3-none-any.whl
Size 8.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
0dde35a10df70167bde81a5d116ef3891c28c1a5cc6a0facaf19ead3f801d97c
BLAKE2b-256 checksum
How to use checksums
ca380215f3884ea64625f9ca26ce648eb62ee5e84e2f31e14523af7410a2b137
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via RestSharp/106.13.0.0

Security for agent-framework-nvidia 1.0.0a261030

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page