openembedforge 0.2.0a1

Forge specialized retrieval models from your own data.

OpenEmbedForge turns domain corpora into auditable retrieval-training datasets and specialized retrievers for RAG.

Not another vector database. Not another RAG orchestration framework. Not just another embedding-model wrapper.

OpenEmbedForge focuses on:

corpus -> training data -> specialized retrieval model -> evaluation -> deployment

Quick Start

Install the local development package:

pip install -e ".[dev]"

Use the v0.1 Python API:

from openembedforge.corpus import WordChunker, chunk_documents, load_documents
from openembedforge.data import dataset_from_chunks
from openembedforge.mining import RandomNegativeMiner

documents = list(load_documents("./examples/data"))
chunks = chunk_documents(documents, WordChunker(max_words=64, overlap_words=8))
dataset = RandomNegativeMiner(negatives_per_example=1, seed=42).mine(dataset_from_chunks(chunks))

dataset.to_jsonl(".tmp/train.jsonl")

Or run the same MVP flow from the CLI:

openembedforge corpus prepare examples/data --output .tmp/corpus.jsonl
openembedforge dataset make .tmp/corpus.jsonl --output .tmp/train.jsonl
openembedforge train --dataset .tmp/train.jsonl --output .tmp/model
openembedforge evaluate --model .tmp/model --dataset .tmp/train.jsonl

For a raw corpus path:

openembedforge train ./documents --output ./domain-model

Installation

Runtime is currently stdlib-only. Development tooling is optional:

pip install -e ".[dev]"

Future extras are reserved for heavier stacks:

pip install "openembedforge[train]"
pip install "openembedforge[onnx]"

openembedforge[train] enables optional Hugging Face Transformers backends, including CLIP/SigLIP-style multimodal encoders:

openembedforge train \
  --dataset .tmp/train.jsonl \
  --backend transformer \
  --model-name-or-path sentence-transformers/all-MiniLM-L6-v2 \
  --output .tmp/transformer-model

The default test suite builds a tiny local Transformer model, so CI does not download models.

Architecture

The current MVP includes:

• corpus loading for .txt, .md, .json, and .jsonl
• deterministic word-window chunking
• canonical RetrievalExample and JSONL datasets
• deterministic easy-negative mining
• a dependency-free dense hashing encoder for local tests and examples
• optional Transformer encoder training with an in-batch InfoNCE objective
• Recall@K, MRR, and nDCG@10
• CLI and Python APIs

Hard-negative mining, distillation, Matryoshka, ONNX, sparse, and late-interaction implementations are future work.

Multimodal / Vision RAG

OpenEmbedForge now includes early multimodal foundations for image-text retrieval research. The goal is to test whether adapting a retriever to a specialised domain corpus improves evidence retrieval over generic embeddings, without making hosted APIs, CUDA, or vector databases mandatory.

flowchart LR
  A[Domain image/text corpus] --> B[Training-data generation]
  B --> C[Multimodal contrastive training]
  C --> D[Specialised retriever]
  D --> E[Dense / multivector retrieval]
  E --> F[Context selection]
  F --> G[Vision-language model]
  G --> H[Grounded answer]

The current stable layer provides:

• MultimodalCorpus.from_csv(...) for paired image-text CSV files
• MultimodalCorpus.from_huggingface(...) for Dataset-like image-text rows
• MultimodalSample with metadata preservation
• deterministic train/validation/test splits, including group-aware splits
• in-memory dense retrieval for text->image, image->text, image->image, and text->text
• optional TransformersMultimodalEncoder for CLIP/SigLIP-style Hugging Face models
• Recall@K, MRR, mAP, and nDCG retrieval evaluation by direction
• random and metadata-aware negative miners that avoid obvious false negatives
• provider-neutral MultimodalEncoder and VisionLanguageGenerator protocols
• RAG context selectors: top-k, score threshold, and metadata diversity
• experimental pure-PyTorch MaxSim and patch-region utilities in openembedforge.experimental.multivector

Example:

from openembedforge.multimodal import (
    DenseMultimodalRetriever,
    HashingMultimodalEncoder,
    MultimodalCorpus,
    TransformersMultimodalEncoder,
)

corpus = MultimodalCorpus.from_csv(
    "metadata.csv",
    image_column="image_path",
    text_column="report",
    group_column="patient_id",
)
retriever = DenseMultimodalRetriever(HashingMultimodalEncoder(), corpus)
results = retriever.search_text("right-sided pleural effusion", top_k=5)

CLI:

openembedforge multimodal inspect metadata.csv --text-column report --group-column patient_id
openembedforge multimodal train metadata.csv \
  --text-column report \
  --base-model hashing \
  --output ./outputs/domain-model
openembedforge multimodal evaluate metadata.csv --text-column report --model hashing --json
openembedforge multimodal benchmark metadata.csv \
  --text-column report \
  --baseline google/siglip-base-patch16-224 \
  --model ./outputs/domain-model
openembedforge multimodal search metadata.csv --text-column report --query "right effusion"

Research inspiration: the architecture is influenced by CLIP/SigLIP-style dual encoders, domain-specific vision-language pretraining such as MedCLIP and BiomedCLIP, ColBERT-style late interaction, ColPali-family visual retrieval, multimodal RAG, and region-level visual retrieval. OpenEmbedForge does not claim to implement those systems unless a module says so.

Medical imaging is treated as an evaluation use case, not a hard-coded assumption. This software is a research/development tool and is not a medical device or clinical diagnostic system.

Roadmap

v0.2: synthetic query generation, query quality filtering, BM25/dense hard negatives, iterative mining, full provenance.

v0.3: teacher/student distillation, domain-specialized small encoders, Matryoshka dimensions, ONNX, quantization.

v0.4: continued domain pretraining, sparse retrieval, late interaction, hybrid retrieval, vector-store adapters.

v0.5: AutoRetriever, deployment-aware strategy selection, RAG-aware evaluation, experimental scratch pretraining.

CLI

openembedforge --help
openembedforge version
openembedforge corpus prepare ./documents --output ./data/corpus.jsonl
openembedforge dataset make ./data/corpus.jsonl --output ./data/train.jsonl
openembedforge dataset inspect ./data/train.jsonl
openembedforge train --dataset ./data/train.jsonl --output ./models/domain-retriever
openembedforge evaluate --model ./models/domain-retriever --dataset ./data/train.jsonl

Evaluation

Evaluation is independent of training code and reports:

• Recall@1
• Recall@5
• Recall@10
• MRR
• mAP
• nDCG@10

JSON output is available:

openembedforge evaluate --model .tmp/model --dataset .tmp/train.jsonl --json

Documentation

• Branding
• Web corpus training notebook
• Roadmap
• Research questions

Contributing

See CONTRIBUTING.md.

License

Apache-2.0.