Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

Get Started with Microsoft Agent Framework NVIDIA NIM

An alpha NVIDIA NIM embedding integration for Microsoft Agent Framework. Please install this package:

pip install agent-framework-nvidia --pre

and see the README for more information.

See the NVIDIA samples for a runnable example.

Embedding Client

The NvidiaEmbeddingClient provides embedding generation using models hosted on NVIDIA NIM. NIM exposes an OpenAI-compatible surface, so this client builds on the OpenAI embedding client rather than reimplementing the wire format.

Quick Start

from agent_framework_nvidia import NvidiaEmbeddingClient

# Using environment variables (NVIDIA_API_KEY, NVIDIA_EMBEDDING_MODEL)
# Parameters can also be passed directly:
# NvidiaEmbeddingClient(model="nvidia/nemotron-3-embed-1b", api_key="your-api-key")
client = NvidiaEmbeddingClient()

result = await client.get_embeddings(["Hello, world!", "How are you?"])
for embedding in result:
    print(f"Dimensions: {embedding.dimensions}")
    print(f"Vector: {embedding.vector[:5]}...")

# Close the HTTP client the embedding client created. A client passed as async_client is left open.
await client.close()

Configuration

Environment Variable Description
NVIDIA_API_KEY Your NVIDIA API key, from build.nvidia.com
NVIDIA_EMBEDDING_MODEL Embedding model name (e.g., nvidia/nemotron-3-embed-1b)
NVIDIA_BASE_URL Optional endpoint override, for a self-hosted NIM container

The service throttles bursts with 429, which the OpenAI SDK retries twice by default. For heavy parallel ingestion, pass a pre-configured async_client, for example AsyncOpenAI(..., max_retries=5), to change retries or timeouts.

Options

NvidiaEmbeddingOptions adds two NVIDIA fields to the OpenAI embedding options:

Option Values Purpose
input_type "passage", "query" Asymmetric retrieval models embed documents and search text differently
truncate "NONE", "START", "END" Reject over-long input, or drop tokens from that end

For retrieval, embed stored documents as passages and search text as queries. Vector collections forward embeddings_options to each request:

await collection.upsert(notes, embeddings_options={"input_type": "passage", "truncate": "END"})
results = await collection.search("How do I call a model?", embeddings_options={"input_type": "query"})

create_vector_search_tool(..., embeddings_options={"input_type": "query"}) does the same for the tool an agent calls. nvidia/nemotron-3-embed-1b accepts only 2048 dimensions, so declare that on the vector field.

The hosted endpoint accepts at most 256 inputs and a 2 MiB body per request, so the client sends larger calls, such as an upsert of many records, as consecutive requests within both limits and returns the vectors in order.

Without truncate, the service shortens input over the model's token limit itself, but it rejects any input over 65,536 characters, and one such document fails the whole call, so nothing in that upsert is written. Set truncate="END" (or "START") when ingesting documents of unknown length. truncate="NONE" rejects input over the token limit instead.

Choosing a model

Model availability is per-account: the endpoint lists more embedding models than a given API key is entitled to call, and an unentitled model returns 404 ... Not found for account rather than a permission error. Retired models return 410 Gone. If a model name from the documentation does not work, list the models available to your key rather than assuming the name is wrong. An invalid key returns 403 Forbidden with Authorization failed, not 401.

Metadata

Release files for agent-framework-nvidia 1.0.0a261106

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for agent-framework-nvidia 1.0.0a261106
File Size Uploaded Provenance Actions
agent_framework_nvidia-1.0.0a261106.tar.gz 7.6 kB Without provenance Details for agent_framework_nvidia-1.0.0a261106.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for agent-framework-nvidia 1.0.0a261106
File Interpreter ABI Platform Actions
agent_framework_nvidia-1.0.0a261106-py3-none-any.whl Python 3 none any Details for agent_framework_nvidia-1.0.0a261106-py3-none-any.whl

Total release size: 16.5 kB

Release files / agent_framework_nvidia-1.0.0a261106.tar.gz

Download URL agent_framework_nvidia-1.0.0a261106.tar.gz
Size 7.6 kB
Tags Source
SHA-256 checksum
How to use checksums
da293207b4f8bc7c8ec2b98c5f6d888aab596b0392c8e90f9cb8a652499db41e
BLAKE2b-256 checksum
How to use checksums
3e0a95586d9d95763af6315519b37b23b96dee646e8ab874d5118e8ba6d8596a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via RestSharp/106.13.0.0

Release files / agent_framework_nvidia-1.0.0a261106-py3-none-any.whl

Download URL agent_framework_nvidia-1.0.0a261106-py3-none-any.whl
Size 8.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
7a341b02436f6b921f3fdaa3483c1df421f3d85a24f6b373a7f09be87d94c217
BLAKE2b-256 checksum
How to use checksums
0459b40d88149f518df6d4169ba9c597693595dabc053accc046297235ba2aca
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via RestSharp/106.13.0.0

Security for agent-framework-nvidia 1.0.0a261106

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page