just-semantic-search
LLM-agnostic semantic-search library with hybrid search support and multiple backends.
Features
- 🔍 Hybrid search combining semantic and keyword search
- 🚀 Multiple backend support (Meilisearch, more coming soon)
- 📄 Smart document splitting with semantic awareness
- 🔌 LLM-agnostic - works with any embedding model
- 🎯 Optimized for scientific and technical content
- 🛠 Easy to use API and CLI tools
Installation
Make sure you have at least Python 3.11 installed.
Using pip
pip install just-semantic-search # Core package
pip install just-semantic-search-meili # Meilisearch backend
Using Poetry
poetry add just-semantic-search # Core package
poetry add just-semantic-search-meili # Meilisearch backend
From Source
# Install Poetry if you haven't already
curl -sSL https://install.python-poetry.org | python3 -
# Clone the repository
git clone https://github.com/your-username/just-semantic-search.git
cd just-semantic-search
# Install dependencies and create virtual environment
poetry install
# Activate the virtual environment
poetry shell
Docker Setup for Meilisearch
The project includes a Docker Compose configuration for running Meilisearch. Simply run:
./bin/meili.sh
This will start a Meilisearch instance with vector search enabled and persistent data storage.
Quick Start
Document Splitting
from just_semantic_search.article_semantic_splitter import ArticleSemanticSplitter
from sentence_transformers import SentenceTransformer
# Initialize model and splitter
model = SentenceTransformer('thenlper/gte-base')
splitter = ArticleSemanticSplitter(model)
# Split document with metadata
documents = splitter.split_file(
"path/to/document.txt",
embed=True,
title="Document Title",
source="https://source.url"
)
Hybrid Search with Meilisearch
from just_semantic_search.meili.rag import MeiliConfig, MeiliRAG
# Configure Meilisearch
config = MeiliConfig(
host="127.0.0.1",
port=7700,
api_key="your_api_key"
)
# Initialize RAG
rag = MeiliRAG(
"test_index",
"thenlper/gte-base",
config,
create_index_if_not_exists=True
)
# Add documents and search
rag.add_documents_sync(documents)
results = rag.search(
text_query="What are CAD-genes?",
vector=model.encode("What are CAD-genes?")
)
Project Structure
The project consists of multiple components:
core: Core interfaces for hybrid search implementationsmeili: Meilisearch backend implementation
Contributing
Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.
License
This project is licensed under the Apache 2.0 License - see the LICENSE file for details.
Citation
If you use this software in your research, please cite:
@software{just_semantic_search,
title = {just-semantic-search: LLM-agnostic semantic search library},
author = {Karmazin, Alex and Kulaga, Anton},
year = {2024},
url = {https://github.com/your-username/just-semantic-search}
}
Metadata
Release files for just-semantic-search 0.4.8
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| just_semantic_search-0.4.8.tar.gz | 23.4 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| just_semantic_search-0.4.8-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 54.9 kB
Release files / just_semantic_search-0.4.8.tar.gz
| Download URL | just_semantic_search-0.4.8.tar.gz |
|---|---|
| Size | 23.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
313e24c15ab1a0cac13e22b23e40620224826b7190b22fca4c4ee64d87ffd8a4
|
|
BLAKE2b-256 checksum How to use checksums |
6acf32c9dab84e0197c4c5ed9e5461b94df02d616cbc5518d558edd3734aec71
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.2.1 CPython/3.10.12 Linux/5.15.0-161-generic
|
Release files / just_semantic_search-0.4.8-py3-none-any.whl
| Download URL | just_semantic_search-0.4.8-py3-none-any.whl |
|---|---|
| Size | 31.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
737a2f31eb96ea6591261d6b7cbc2f4cd7f6d65e385bd63721ea6b6d37afb818
|
|
BLAKE2b-256 checksum How to use checksums |
a308bb40c5724255cee1399776f151d0d0ce409c412529ab580ce880e6b5c62b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.2.1 CPython/3.10.12 Linux/5.15.0-161-generic
|