langchain-dolphindb 0.0.1

Langchain integration for DolphinDB Vector Store

langchain-dolphindb

langchain-dolphindb integrates DolphinDB's high-performance database capabilities into LangChain's vector store ecosystem, enabling efficient storage, retrieval, and semantic search of vector data.

Installation

Required version: DolphinDB 3.00.0 or higher & dolphindb 3.0.0 or higher

Install the langchain-dolphindb with the following pip command:

pip install -U langchain-dolphindb

Manage Vectorstore

Connect to vector store

Configure DolphinDBConfig to connect to DolphinDB and create the vector store if it doesn’t exist.

The following example uses DeterministicFakeEmbedding as the embeddings.

from langchain_community.embeddings import DeterministicFakeEmbedding
from langchain_dolphindb import DolphinDBConfig, DolphinDBVectorStore

embeddings = DeterministicFakeEmbedding(size=4096)

config = DolphinDBConfig(
    host="localhost", port=8848, username="admin", password="123456"
)

vector_store = DolphinDBVectorStore(embedding=embeddings, config=config)

Delete vector store

You can use DolphinDBVectorStore.conn to control the connection to the vector store and delete it with DolphinDBVectorStore.conn.drop_database method.

vector_store.conn.drop_database()

Add items to vector store

You can add items to a vector store by using the DolphinDBVectorStore.add_documents method.

from langchain_core.documents import Document

docs = [
    Document(
        page_content="there are cats in the pond",
        metadata={"id": 1, "location": "pond", "topic": "animals"},
    ),
    Document(
        page_content="ducks are also found in the pond",
        metadata={"id": 2, "location": "pond", "topic": "animals"},
    ),
    Document(
        page_content="fresh apples are available at the market",
        metadata={"id": 3, "location": "market", "topic": "food"},
    ),
    Document(
        page_content="the market also sells fresh oranges",
        metadata={"id": 4, "location": "market", "topic": "food"},
    ),
    Document(
        page_content="the new art exhibit is fascinating",
        metadata={"id": 5, "location": "museum", "topic": "art"},
    ),
    Document(
        page_content="a sculpture exhibit is also at the museum",
        metadata={"id": 6, "location": "museum", "topic": "art"},
    ),
    Document(
        page_content="a new coffee shop opened on Main Street",
        metadata={"id": 7, "location": "Main Street", "topic": "food"},
    ),
    Document(
        page_content="the book club meets at the library",
        metadata={"id": 8, "location": "library", "topic": "reading"},
    ),
    Document(
        page_content="the library hosts a weekly story time for kids",
        metadata={"id": 9, "location": "library", "topic": "reading"},
    ),
    Document(
        page_content="a cooking class for beginners is offered at the community center",
        metadata={"id": 10, "location": "community center", "topic": "classes"},
    ),
]

vector_store.add_documents(docs, ids=[doc.metadata["id"] for doc in docs])

Delete items from vector store

You can delete items from a vector store by using the DolphinDBVectorStore.delete method.

vector_store.delete(ids=["3"])

Query vector store

Once your vector store has been created and the relevant documents have been added you will most likely wish to query it during the running of your chain or agent.

Query directly

You can use jsonExtract function to extract metadata columns.

results = vector_store.similarity_search(
    "kitty", k=10, where_str="""jsonExtract(metadata, "id", "int") in [1,5,2,9]"""
)
for doc in results:
    print(f"* {doc.page_content} [{doc.metadata}]")

Query by turning into retriever

You can also transform the vector store into a retriever.

retriever = vector_store.as_retriever(search_type="similarity", search_kwargs={"k": 1})
results = retriever.invoke("kitty")
for doc in results:
    print(f"* {doc.page_content} [{doc.metadata}]")