Rakam System Vectorstore

The vectorstore package of Rakam Systems providing vector database solutions and document processing capabilities.

Overview

rakam-systems-vectorstore provides comprehensive vector storage, embedding models, and document loading capabilities. This package depends on rakam-systems-core.

Features

• Configuration-First Design: Change your entire vector store setup via YAML — no code changes
• Multiple Backends: PostgreSQL with pgvector and FAISS in-memory storage
• Flexible Embeddings: SentenceTransformers, OpenAI, and Cohere
• Document Loaders: PDF, DOCX, HTML, Markdown, CSV, and more
• Search Capabilities: Vector search, keyword search (BM25), and hybrid search
• Chunking: Intelligent text chunking with context preservation

Installation

pip install rakam-systems-vectorstore

# With specific backends
pip install rakam-systems-vectorstore[postgres]
pip install rakam-systems-vectorstore[faiss]
pip install rakam-systems-vectorstore[all]

Available extras:

Quick Start

from rakam_systems_vectorstore import FaissStore, Node, NodeMetadata

store = FaissStore(
    name="my_store",
    base_index_path="./indexes",
    embedding_model="Snowflake/snowflake-arctic-embed-m",
    initialising=True
)

nodes = [
    Node(
        content="Python is great for AI",
        metadata=NodeMetadata(source_file_uuid="doc1", position=0)
    )
]

store.create_collection_from_nodes("my_collection", nodes)
results, _ = store.search(collection_name="my_collection", query="AI programming", number=5)

PostgreSQL store (Django, FastAPI, or plain Python)

The PostgreSQL store is exposed at a stable import path:

from rakam_systems_vectorstore.stores import (
    ConfigurablePgVectorStore,
    VectorStoreConfig,
)

store = ConfigurablePgVectorStore(config=VectorStoreConfig())
store.setup()  # creates the pgvector extension and tables if needed

The store connects through Django's ORM. Inside a Django process it uses your project's existing database configuration. Outside Django (FastAPI, workers, scripts) importing from rakam_systems_vectorstore.stores automatically configures Django from the POSTGRES_* environment variables — no DJANGO_SETTINGS_MODULE or django.setup() needed on your side. A host that has already configured Django is never overridden.

To target a database other than the POSTGRES_* defaults, configure it explicitly before the first import from the store:

from rakam_systems_vectorstore.config import DatabaseConfig
from rakam_systems_vectorstore.stores import ensure_django_configured

ensure_django_configured(DatabaseConfig(host="db.internal", database="tickets"))

from rakam_systems_vectorstore.stores import ConfigurablePgVectorStore

Core Components

• ConfigurablePgVectorStore — PostgreSQL with pgvector, hybrid search, keyword search
• FaissStore — In-memory FAISS-based vector search
• ConfigurableEmbeddings — SentenceTransformers, OpenAI, Cohere backends
• AdaptiveLoader — Auto-detects and loads PDF, DOCX, HTML, Markdown, CSV, email, code
• TextChunker / AdvancedChunker — Sentence-based and context-aware chunking

Environment Variables

Documentation

For PostgreSQL setup, search examples, YAML configuration, and full API reference, see the official documentation.

License

Apache 2.0