endee-llamaindex 0.1.2

Vector Database for Fast ANN Searches

Endee LlamaIndex Integration

This package provides an integration between Endee (a vector database) and LlamaIndex, allowing you to use Endee as a vector store backend for LlamaIndex.

Features

• Vector Storage: Use Endee for your LlamaIndex embeddings
• Multiple Distance Metrics: Support for cosine, L2, and inner product distance metrics
• Metadata Filtering: Filter search results based on metadata
• High Performance: Optimized for speed and efficiency

Installation

pip install endee-llamaindex

This will install both the endee-llamaindex package and its dependencies (endee and llama-index).

Quick Start

import os
from llama_index.core.schema import TextNode
from llama_index.core.vector_stores.types import VectorStoreQuery
from endee_llamaindex import EndeeVectorStore

# Configure your Endee credentials
api_token = os.environ.get("ENDEE_API_TOKEN")
index_name = "my_llamaindex_vectors"
dimension = 1536  # OpenAI ada-002 embedding dimension

# Initialize the vector store
vector_store = EndeeVectorStore.from_params(
    api_token=api_token,
    index_name=index_name,
    dimension=dimension,
    space_type="cosine"
)

# Create a node with embedding
node = TextNode(
    text="This is a sample document",
    id_="doc1",
    embedding=[0.1, 0.2, 0.3, ...],  # Your embedding vector
    metadata={
        "doc_id": "doc1",
        "source": "example",
        "author": "Endee"
    }
)

# Add the node to the vector store
vector_store.add([node])

# Query the vector store
query = VectorStoreQuery(
    query_embedding=[0.2, 0.3, 0.4, ...],  # Your query vector
    similarity_top_k=5
)

results = vector_store.query(query)

# Process results
for node, score in zip(results.nodes, results.similarities):
    print(f"Node ID: {node.node_id}, Similarity: {score}")
    print(f"Text: {node.text}")
    print(f"Metadata: {node.metadata}")

Using with LlamaIndex

from llama_index.core import VectorStoreIndex, StorageContext
from llama_index.embeddings.openai import OpenAIEmbedding

# Initialize your nodes or documents
nodes = [...]  # Your nodes with text but no embeddings yet

# Setup embedding function
embed_model = OpenAIEmbedding()  # Or any other embedding model

# Initialize Endee vector store
vector_store = EndeeVectorStore.from_params(
    api_token=api_token,
    index_name=index_name,
    dimension=1536,  # Make sure this matches your embedding dimension
)

# Create storage context
storage_context = StorageContext.from_defaults(vector_store=vector_store)

# Create vector index
index = VectorStoreIndex(
    nodes, 
    storage_context=storage_context,
    embed_model=embed_model
)

# Query the index
query_engine = index.as_query_engine()
response = query_engine.query("Your query here")
print(response)

Configuration Options

The EndeeVectorStore constructor accepts the following parameters:

• api_token: Your Endee API token
• index_name: Name of the Endee index
• dimension: Vector dimension (required when creating a new index)
• space_type: Distance metric, one of "cosine", "l2", or "ip" (default: "cosine")
• batch_size: Number of vectors to insert in a single API call (default: 100)
• text_key: Key to use for storing text in metadata (default: "text")
• remove_text_from_metadata: Whether to remove text from metadata (default: False)