LoLLMsVectorDB
LoLLMsVectorDB: A modular text-based database manager for retrieval-augmented generation (RAG), seamlessly integrating with the LoLLMs ecosystem. Supports various vectorization methods and directory bindings for efficient text data management.
Features
- Flexible Vectorization: Supports multiple vectorization methods including TF-IDF and Word2Vec.
- Directory Binding: Automatically updates the vector store with text data from a specified directory.
- Efficient Search: Provides fast and accurate search results with metadata to locate the original text chunks.
- Modular Design: Easily extendable to support new vectorization methods and functionalities.
Installation
pip install lollmsvectordb
Usage
Example with TFIDFVectorizer
from lollmsvectordb import TFIDFVectorizer, VectorDatabase, DirectoryBinding
# Initialize the vectorizer
tfidf_vectorizer = TFIDFVectorizer()
tfidf_vectorizer.fit(["This is a sample text.", "Another sample text."])
# Create the vector database
db = VectorDatabase("vector_db.sqlite", tfidf_vectorizer)
# Bind a directory to the vector database
directory_binding = DirectoryBinding("path_to_your_directory", db)
# Update the vector store with text data from the directory
directory_binding.update_vector_store()
# Search for a query in the vector database
results = directory_binding.search("This is a sample text.")
print(results)
Adding New Vectorization Methods
To add a new vectorization method, create a subclass of the Vectorizer class and implement the vectorize method.
from lollmsvectordb import Vectorizer
class CustomVectorizer(Vectorizer):
def vectorize(self, data):
# Implement your custom vectorization logic here
pass
Contributing
Contributions are welcome! Please fork the repository and submit a pull request.
License
This project is licensed under the MIT License.
Contact
For any questions or suggestions, feel free to reach out to the author:
- Twitter: @ParisNeo_AI
- Discord: Join our Discord
- Sub-Reddit: r/lollms
- Instagram: spacenerduino
Acknowledgements
Special thanks to the LoLLMs community for their continuous support and contributions.
1 maintainer
Release files for lollmsvectordb 1.3.8
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| lollmsvectordb-1.3.8.tar.gz | 36.7 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| lollmsvectordb-1.3.8-py3-none-any.whl | Python 3 | none | any | Details |
Total release size:84.4 kB
Release files / lollmsvectordb-1.3.8.tar.gz
| Download URL | lollmsvectordb-1.3.8.tar.gz |
|---|---|
| Size | 36.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
2a1284486ae76cc3015c710001e091a91556e797e494278926b4c66022cb8590
|
|
BLAKE2b-256 checksum How to use checksums |
4df08bfbdac430da866b53e9af91cba810d4334819f82c535a8e073e914e5e46
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.0.1 CPython/3.12.4
|
Release files / lollmsvectordb-1.3.8-py3-none-any.whl
| Download URL | lollmsvectordb-1.3.8-py3-none-any.whl |
|---|---|
| Size | 47.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
bd1a83db70e9707d9bf3f60d48f91f326861810c59d6249bd6b8add6410851f7
|
|
BLAKE2b-256 checksum How to use checksums |
b86975ae081e97a00ac08156fdcb88725053c5ce4bfadd10590958ee68cb404f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.0.1 CPython/3.12.4
|
