endee-langchain 0.1.0

High Speed Vector Database for Faster and Efficient ANN Searches with LangChain

Endee LangChain Integration

This package provides an integration between Endee (a high speed vector database) and LangChain, allowing you to use Endee as a vector store backend for LangChain.

Features

• Multiple Distance Metrics: Support for cosine, L2, and inner product distance metrics
• Metadata Filtering: Filter search results based on metadata
• High Performance: Optimized for speed and efficiency with vector data

Installation

pip install endee-langchain

This will install both the endee-langchain package and its dependencies (endee, langchain, and langchain-core).

Quick Start

import os
from langchain_openai import OpenAIEmbeddings
from langchain_core.documents import Document
from endee.endee_client import Endee
from endee_langchain import EndeeVectorStore

# Configure your Endee credentials
api_token = os.environ.get("ENDEE_API_TOKEN")
nd = Endee(token=api_token)

# Initialize embedding model
embedding_model = OpenAIEmbeddings()

# Initialize the vector store
vector_store = EndeeVectorStore.from_params(
    embedding=embedding_model,
    api_token=api_token,
    index_name="my_langchain_vectors",
    space_type="cosine"
)

# Add documents
texts = [
    "Endee is the world's fastest vector database",
    "LangChain is a framework for developing applications powered by language models",
    "Vector databases store vector embeddings and enable fast similarity search."
]

metadatas = [
    {"source": "product", "category": "database"},
    {"source": "github", "category": "framework"},
    {"source": "textbook", "category": "security"}
]

vector_store.add_texts(texts=texts, metadatas=metadatas)

# Search similar documents
results = vector_store.similarity_search("How do vector databases work?", k=2)

# Process results
for doc in results:
    print(f"Content: {doc.page_content}")
    print(f"Metadata: {doc.metadata}")
    print()

Filtering Search Results

You can filter search results based on metadata using flexible query operators. Here's an example using a filter:

Search with a filter

query = "Tell me about Endee"
filter_dict = {"category": {"$eq": "database"}}
 
filtered_results = vector_store.similarity_search(
    query=query,
    k=3,
    filter=filter_dict
)

print(f"Query: '{query}' with filter: {filter_dict}")
print(f"\nFound {len(filtered_results)} filtered results:")
for i, doc in enumerate(filtered_results):
    print(f"\nResult {i+1}:")
    print(f"Content: {doc.page_content}")
    print(f"Metadata: {doc.metadata}")

Supported Filter Operators

• $eq: Matches records with metadata values equal to a specified value
  Example:

  {
    "category": { "$eq": "database" }
  }
  

• $in: Matches records with metadata values that are in a specified array
  Example:

  {
    "category": { "$in": ["database", "framework"] }
  }
  

• $range: Matches numeric metadata fields within a given range
  Format: [min, max]
  Example:

  {
    "score": { "$range": [0, 10] }
  }
  

Using with LangChain

Endee can be used anywhere a LangChain vector store is needed:

from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from endee_langchain import EndeeVectorStore

# Initialize your vector store
vector_store = EndeeVectorStore.from_params(
    embedding=OpenAIEmbeddings(),
    api_token="your_api_token",
    index_name="your_index_name"
)

# Create a retriever
retriever = vector_store.as_retriever()

# Create the RAG chain
model = ChatOpenAI()
prompt = ChatPromptTemplate.from_template(
    """Answer the following question based on the provided context:
    
    Context: {context}
    Question: {question}
    """
)

rag_chain = (
    {"context": retriever, "question": RunnablePassthrough()}
    | prompt
    | model
    | StrOutputParser()
)

# Use the chain
response = rag_chain.invoke("What is Endee?")
print(response)

API Reference

EndeeVectorStore

The main class for integrating with LangChain. Key methods include:

• __init__: Initialize with a Endee index or parameters to create a new one
• from_params: Create a vector store using an API token
• add_texts: Add text documents with optional metadata
• similarity_search: Search for similar documents
• similarity_search_with_score: Search and return similarity scores
• delete: Delete documents by ID or filter

Configuration Options

The EndeeVectorStore constructor and from_params method accept the following parameters:

• embedding: LangChain embedding function to use
• api_token: Your Endee API token
• index_name: Name of the Endee index
• dimension: Vector dimension (can be inferred from embedding model)
• space_type: Distance metric, one of "cosine", "l2", or "ip" (default: "cosine")
• text_key: Key to use for storing text in metadata (default: "text")