nlweb-retrieval

Bundle package containing all retrieval providers for NLWeb.

Included Providers

This package includes clients for:

• Azure AI Search - Azure's vector search service with managed identity support
• Elasticsearch - Popular open-source search and analytics engine
• Milvus - Open-source vector database
• Qdrant - Vector similarity search engine
• PostgreSQL (pgvector) - PostgreSQL with vector extension
• OpenSearch - Open-source search and analytics suite
• Snowflake Cortex Search - Snowflake's vector search
• Shopify MCP - Shopify Model Context Protocol
• Cloudflare AutoRAG - Cloudflare's RAG service
• Bing Search - Microsoft Bing web search

Installation

pip install nlweb-core nlweb-retrieval

Configuration Example

Create config.yaml:

retrieval:
  provider: elasticsearch
  import_path: nlweb_retrieval.elasticsearch_client
  class_name: ElasticsearchClient
  api_endpoint_env: ELASTICSEARCH_URL
  index_name: my_index
  use_knn: true

Azure AI Search Example

retrieval:
  provider: azure_ai_search
  import_path: nlweb_retrieval.azure_search_client
  class_name: AzureSearchClient
  api_endpoint_env: AZURE_SEARCH_ENDPOINT
  auth_method: azure_ad  # or api_key
  index_name: my-search-index

Qdrant Example

retrieval:
  provider: qdrant
  import_path: nlweb_retrieval.qdrant
  class_name: QdrantVectorClient
  database_path: ./data/qdrant_db  # for local storage
  # OR for remote:
  # api_endpoint_env: QDRANT_URL
  # api_key_env: QDRANT_API_KEY

Usage

import nlweb_core

# Initialize with config
nlweb_core.init(config_path="./config.yaml")

# Use retrieval
from nlweb_core import retriever

# Search
results = await retriever.search(
    query="example query",
    site="example.com",
    num_results=10
)

Provider Import Paths

Use these in your config file:

License

MIT License - Copyright (c) 2025 Microsoft Corporation