vecx-llamaindex 0.1.2

Encrypted Vector Database for Secure and Fast ANN Searches

VectorX LlamaIndex Integration

This package provides an integration between VectorX (an encrypted vector database) and LlamaIndex, allowing you to use VectorX as a vector store backend for LlamaIndex.

Features

• Encrypted Vector Storage: Use VectorX's client-side encryption for your LlamaIndex embeddings
• Multiple Distance Metrics: Support for cosine, L2, and inner product distance metrics
• Metadata Filtering: Filter search results based on metadata
• High Performance: Optimized for speed and efficiency with encrypted data

Installation

pip install vecx-llamaindex

This will install both the vecx-llamaindex package and its dependencies (vecx and llama-index).

Quick Start

import os
from llama_index.core.schema import TextNode
from llama_index.core.vector_stores.types import VectorStoreQuery
from vecx_llamaindex import VectorXVectorStore

# Configure your VectorX credentials
api_token = os.environ.get("VECTORX_API_TOKEN")
encryption_key = os.environ.get("VECTORX_ENCRYPTION_KEY")  # or generate a new one
index_name = "my_llamaindex_vectors"
dimension = 1536  # OpenAI ada-002 embedding dimension

# Initialize the vector store
vector_store = VectorXVectorStore.from_params(
    api_token=api_token,
    encryption_key=encryption_key,
    index_name=index_name,
    dimension=dimension,
    space_type="cosine"
)

# Create a node with embedding
node = TextNode(
    text="This is a sample document",
    id_="doc1",
    embedding=[0.1, 0.2, 0.3, ...],  # Your embedding vector
    metadata={
        "doc_id": "doc1",
        "source": "example",
        "author": "VectorX"
    }
)

# Add the node to the vector store
vector_store.add([node])

# Query the vector store
query = VectorStoreQuery(
    query_embedding=[0.2, 0.3, 0.4, ...],  # Your query vector
    similarity_top_k=5
)

results = vector_store.query(query)

# Process results
for node, score in zip(results.nodes, results.similarities):
    print(f"Node ID: {node.node_id}, Similarity: {score}")
    print(f"Text: {node.text}")
    print(f"Metadata: {node.metadata}")

Using with LlamaIndex

from llama_index.core import VectorStoreIndex, StorageContext
from llama_index.embeddings.openai import OpenAIEmbedding

# Initialize your nodes or documents
nodes = [...]  # Your nodes with text but no embeddings yet

# Setup embedding function
embed_model = OpenAIEmbedding()  # Or any other embedding model

# Initialize VectorX vector store
vector_store = VectorXVectorStore.from_params(
    api_token=api_token,
    encryption_key=encryption_key,
    index_name=index_name,
    dimension=1536,  # Make sure this matches your embedding dimension
)

# Create storage context
storage_context = StorageContext.from_defaults(vector_store=vector_store)

# Create vector index
index = VectorStoreIndex(
    nodes, 
    storage_context=storage_context,
    embed_model=embed_model
)

# Query the index
query_engine = index.as_query_engine()
response = query_engine.query("Your query here")
print(response)

Configuration Options

The VectorXVectorStore constructor accepts the following parameters:

• api_token: Your VectorX API token
• encryption_key: Your encryption key for the index
• index_name: Name of the VectorX index
• dimension: Vector dimension (required when creating a new index)
• space_type: Distance metric, one of "cosine", "l2", or "ip" (default: "cosine")
• batch_size: Number of vectors to insert in a single API call (default: 100)
• text_key: Key to use for storing text in metadata (default: "text")
• remove_text_from_metadata: Whether to remove text from metadata (default: False)