llama-index vector-stores oceanbase integration
Project description
LlamaIndex Vector-Stores Integration: Oceanbase
OceanBase Database is a distributed relational database. It is developed entirely by Ant Group. The OceanBase Database is built on a common server cluster. Based on the Paxos protocol and its distributed structure, the OceanBase Database provides high availability and linear scalability.
OceanBase currently has the ability to store vectors. Users can easily perform the following operations with SQL:
- Create a table containing vector type fields;
- Create a vector index table based on the HNSW algorithm;
- Perform vector approximate nearest neighbor queries;
- ...
Limitations
The vector storage capability of OceanBase is still being enhanced, and currently has the following limitations:
- Cosine vector distance function is not yet supported for vector index. Currently only inner product distance and Euclidean distance are supported, and Euclidean distance is used by default.
- It should be noted that
OceanBase
currently only supports post-filtering (i.e., filtering based on metadata after performing an approximate nearest neighbor search).
Install dependencies
We use pyobvector
to integrate OceanBase vector store into LlamaIndex.
So it is necessary to install it with pip install pyobvector
before starting.
Setup OceanBase
We recommend using Docker to deploy OceanBase:
docker run --name=ob433 -e MODE=slim -p 2881:2881 -d oceanbase/oceanbase-ce:4.3.3.0-100000142024101215
Usage
%pip install llama-index-vector-stores-oceanbase
%pip install llama-index
# choose dashscope as embedding and llm model, your can also use default openai or other model to test
%pip install llama-index-embeddings-dashscope
%pip install llama-index-llms-dashscope
from llama_index.vector_stores.oceanbase import OceanBaseVectorStore
from pyobvector import ObVecClient
client = ObVecClient()
client.perform_raw_text_sql(
"ALTER SYSTEM ob_vector_memory_limit_percentage = 30"
)
# Initialize OceanBaseVectorStore
oceanbase = OceanBaseVectorStore(
client=client,
dim=1536,
drop_old=True,
normalize=True,
)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file llama_index_vector_stores_oceanbase-0.2.0.tar.gz
.
File metadata
- Download URL: llama_index_vector_stores_oceanbase-0.2.0.tar.gz
- Upload date:
- Size: 8.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.8.3 CPython/3.11.10 Darwin/22.3.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8dbdc5958904be83e98057d5b47d051218d2cc4cbe63cfe4e354d57885a1c86f |
|
MD5 | d0fbcf3235ae33ae9bad0b503142a11d |
|
BLAKE2b-256 | f10c74e3b41635a533c3238501b8873c17249cb7d320375973f030eb518eda63 |
File details
Details for the file llama_index_vector_stores_oceanbase-0.2.0-py3-none-any.whl
.
File metadata
- Download URL: llama_index_vector_stores_oceanbase-0.2.0-py3-none-any.whl
- Upload date:
- Size: 8.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.8.3 CPython/3.11.10 Darwin/22.3.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | ae07589965a73053573820ea67a10dc60a3f796e3dacde6a4d9a9e6f58e3e8c8 |
|
MD5 | 811e2d81cab0080cee35aaf35bc51646 |
|
BLAKE2b-256 | 74ab26984f3914b48b393b94fc0fec20187404285d1ac881ed572ae8c328db40 |