LlamaIndex Vector Stores Integration: Tencent Cloud COS Vectors

English | 中文

llama-index-vector-stores-cos integrates Tencent Cloud COS Vectors with LlamaIndex, letting you use COS Vectors as the backend of a VectorStoreIndex for vector insertion, retrieval, and metadata filtering — with zero ops overhead.

• PyPI: llama-index-vector-stores-cos
• Python: >=3.10, <3.14
• License: MIT

---

Features

• 🚀 Plug & play: swap in as your VectorStoreIndex backend in a single line; works with any LlamaIndex QueryEngine / Retriever.
• 🧩 Standard interface: implements BasePydanticVectorStore with full add / query / get_nodes / delete / delete_nodes / clear, plus async a* counterparts.
• 🔎 Rich filtering: supports the standard LlamaIndex MetadataFilters — EQ/NE/GT/GTE/LT/LTE/IN/NIN/IS_EMPTY combined with AND/OR.
• 🛡 Built-in throttling: token-bucket limits writes to 5 put_vectors calls per second to stay safely below service limits.
• ☁️ Fully managed: both the vector bucket and the index are hosted on Tencent Cloud COS Vectors — no self-hosted vector database required.

---

Installation

Install the main package:

pip install llama-index-vector-stores-cos

Install LlamaIndex ecosystem components as needed (examples below use OpenAI):

# Embedding & LLM (install as needed)
pip install llama-index-embeddings-openai llama-index-llms-openai

# Or local Ollama
# pip install llama-index-embeddings-ollama

# Or HuggingFace
# pip install llama-index-embeddings-huggingface

---

Enable COS Vectors

Full guide: Tencent Cloud Vector Bucket Docs (or 腾讯云向量存储桶 in Chinese).

Quick steps:

1. Log in to the Tencent Cloud Console and enable Cloud Object Storage (COS).

2. Enable COS Vectors and create a vector bucket in your target region (e.g. ap-guangzhou).

3. Go to CAM and create (or reuse) an API key pair (SecretId + SecretKey).

4. Grab the vector bucket's dedicated access domain, in the form of:

   vectors.<region>.coslake.com
   # e.g. vectors.ap-guangzhou.coslake.com
   

⚠️ Domain must be the COS Vectors dedicated domain (vectors.<region>.coslake.com), not the regular COS domain cos.<region>.myqcloud.com. Using the wrong one leads to access failures.

Recommended: inject credentials via environment variables:

export COS_SECRET_ID=your_secret_id
export COS_SECRET_KEY=your_secret_key
export OPENAI_API_KEY=your_openai_api_key

---

Embedding model

This document uses OpenAI text-embedding-3-small (1536 dim) as the minimal example. You can swap in OllamaEmbedding, HuggingFaceEmbedding, or any LlamaIndex BaseEmbedding implementation — just make sure the dimension you pass when creating the index matches the embedding model's output dimension.

from llama_index.core import Settings
from llama_index.embeddings.openai import OpenAIEmbedding

Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")  # 1536 dim

---

Quick start

1. Create a new index and get a VectorStore

create_index_from_bucket creates an index in the specified vector bucket and returns a ready-to-use COSVectorStore.

import os
from qcloud_cos import CosConfig
from llama_index.vector_stores.cos import COSVectorStore

config = CosConfig(
    Region="ap-guangzhou",
    SecretId=os.environ["COS_SECRET_ID"],
    SecretKey=os.environ["COS_SECRET_KEY"],
    Domain="vectors.ap-guangzhou.coslake.com",
)

vector_store = COSVectorStore.create_index_from_bucket(
    bucket_name="example-bucket-1250000000",
    index_name="example-index",
    dimension=1536,              # must match the embedding output
    distance_metric="cosine",    # or "euclidean"
    config=config,
)

2. Reuse an existing index

If the index already exists, just construct a COSVectorStore directly:

from qcloud_cos import CosConfig, CosVectorsClient
from llama_index.vector_stores.cos import COSVectorStore

client = CosVectorsClient(CosConfig(
    Region="ap-guangzhou",
    SecretId=os.environ["COS_SECRET_ID"],
    SecretKey=os.environ["COS_SECRET_KEY"],
    Domain="vectors.ap-guangzhou.coslake.com",
))

vector_store = COSVectorStore(
    bucket_name="example-bucket-1250000000",
    index_name="example-index",
    client=client,          # or pass config=CosConfig(...) instead
)

At least one of client / config is required; if both are provided, client wins.

---

Usage examples

Insert vectors

From text (TextNode)

from llama_index.core.schema import TextNode
from llama_index.core import Settings

nodes = [
    TextNode(text="The Oriental Pearl Tower sits by the Huangpu River in Shanghai.", metadata={"city": "shanghai"}),
    TextNode(text="The Forbidden City was the imperial palace of the Ming and Qing dynasties.", metadata={"city": "beijing"}),
]

for n in nodes:
    n.embedding = Settings.embed_model.get_text_embedding(n.get_content())

vector_store.add(nodes)

From documents (SimpleDirectoryReader)

from llama_index.core import VectorStoreIndex, StorageContext, SimpleDirectoryReader

documents = SimpleDirectoryReader("./data").load_data()
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)

Retrieve vectors

Plain similarity search

retriever = index.as_retriever(similarity_top_k=3)
nodes = retriever.retrieve("Where is the Oriental Pearl Tower?")
for n in nodes:
    print(n.score, n.text[:80])

Or use the query engine directly:

query_engine = index.as_query_engine(similarity_top_k=3)
print(query_engine.query("Where is the Oriental Pearl Tower?"))

Metadata-filtered search

from llama_index.core.vector_stores import (
    MetadataFilter,
    MetadataFilters,
    FilterOperator,
    FilterCondition,
)

filters = MetadataFilters(
    filters=[
        MetadataFilter(key="city", value="shanghai", operator=FilterOperator.EQ),
        MetadataFilter(key="tags", value=["landmark", "tourism"], operator=FilterOperator.IN),
    ],
    condition=FilterCondition.AND,   # OR is also supported; NOT is NOT supported
)

retriever = index.as_retriever(similarity_top_k=5, filters=filters)
nodes = retriever.retrieve("What landmarks are in Shanghai?")

Supported operators: EQ / NE / GT / GTE / LT / LTE / IN / NIN / IS_EMPTY. Conditions: AND / OR. See Parameters.

Node management

# Fetch nodes by ID
nodes = vector_store.get_nodes(node_ids=["node-id-1", "node-id-2"])

# Delete nodes by ID
vector_store.delete_nodes(node_ids=["node-id-1"])

# Delete all nodes belonging to a source doc (matched by metadata ref_doc_id)
vector_store.delete(ref_doc_id="doc-id-xxx")

# ⚠️ Destroy the whole index (dangerous — this is NOT a "truncate")
vector_store.clear()

delete_nodes does not support the filters argument; clear() calls delete_index directly.

---

End-to-end LlamaIndex example

A minimal runnable pipeline: load documents → build index → ask questions.

import os

from qcloud_cos import CosConfig
from llama_index.core import (
    Settings,
    SimpleDirectoryReader,
    StorageContext,
    VectorStoreIndex,
)
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.llms.openai import OpenAI
from llama_index.vector_stores.cos import COSVectorStore

# 1. Configure embedding & LLM
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")  # 1536 dim
Settings.llm = OpenAI(model="gpt-4o-mini")

# 2. Connect to COS Vectors
config = CosConfig(
    Region="ap-guangzhou",
    SecretId=os.environ["COS_SECRET_ID"],
    SecretKey=os.environ["COS_SECRET_KEY"],
    Domain="vectors.ap-guangzhou.coslake.com",
)

# 3. Create a new index (use COSVectorStore(...) directly if the index already exists)
vector_store = COSVectorStore.create_index_from_bucket(
    bucket_name="example-bucket-1250000000",
    index_name="llama-index-demo",
    dimension=1536,
    distance_metric="cosine",
    config=config,
)

# 4. Read local docs → insert vectors
documents = SimpleDirectoryReader("./data").load_data()
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)

# 5. Query
query_engine = index.as_query_engine(similarity_top_k=3)
print(query_engine.query("What is this document mainly about?"))

---

Parameters

COSVectorStore(...) constructor

Param	Type	Default	Description
bucket_name	str	required	COS vector bucket name.
index_name	str	required	Index name.
data_type	str	"float32"	Vector data type.
insert_batch_size	int	500	Max items per write batch; upper bound is 500, otherwise ValueError.
text_field	Optional[str]	None	If set, restore TextNode.text from this metadata key; otherwise use the default serializer.
distance_metric	str	"cosine"	Distance metric; supports "cosine" and "euclidean".
client	Optional[CosVectorsClient]	None	Pre-built client; one of client/config is required.
config	Optional[CosConfig]	None	Used to build the client internally; one of client/config is required.

COSVectorStore.create_index_from_bucket(...) extra params

Param	Type	Default	Description
dimension	int	required	Vector dimension; must match the embedding output.
distance_metric	str	"cosine"	Distance metric for the new index; supports "cosine" and "euclidean".
data_type	str	"float32"	Vector data type of the new index.
insert_batch_size	int	500	Forwarded to the returned COSVectorStore; upper bound is 500.
non_filterable_metadata_keys	Optional[List]	None	Metadata keys that cannot be used as filters; _node_content and _node_type are always appended automatically.
client / config	—	—	Same as the constructor; one of them is required.

Supported filter operators

Operator	Supported	Notes
EQ	✅	equals
NE	✅	not equal
GT/GTE	✅	greater (or eq)
LT/LTE	✅	less (or eq)
IN	✅	value in list
NIN	✅	value not in
IS_EMPTY	✅	field missing
TEXT_MATCH / CONTAINS / others	❌	not supported

Condition	Supported
AND	✅
OR	✅
NOT	❌

Limits & notes

• add writes nodes in batches of insert_batch_size, and the client throttles batches to 5 put_vectors calls per second; large writes will auto-sleep.
• insert_batch_size is capped at 500.
• query only supports VectorStoreQueryMode.DEFAULT; passing HYBRID/SPARSE raises NotImplementedError.
• get_nodes requires node_ids; filters is currently ignored.
• delete_nodes does not support filters; node_ids is required.
• clear() calls delete_index and destroys the whole index — use with care.
• Every core method has an async counterpart: async_add, aquery, aget_nodes, adelete, adelete_nodes, aclear.

---

FAQ

Q1: ValueError: Either 'client' or 'config' must be provided. You passed neither client nor config when constructing COSVectorStore (or calling create_index_from_bucket). Provide at least one.

Q2: ValueError: insert_batch_size must be <= 500 COS Vectors caps batched writes at 500. Lower insert_batch_size.

Q3: ValueError: NOT condition is not supported for COS Vectors filters FilterCondition.NOT is not supported. Rewrite with NE / NIN instead.

Q4: NotImplementedError: Only DEFAULT query mode is supported for COSVectorStore Make sure you're using the default query mode; do not pass VectorStoreQueryMode.HYBRID / SPARSE etc.

Q5: Access fails and returns an XML-formatted error message In most cases, Domain is missing, or it was set to the regular COS domain (for example, cos.<region>.myqcloud.com). Use the COS Vectors dedicated domain vectors.<region>.coslake.com instead.

Q6: What is text_field and when do I need it? By default, COSVectorStore serializes the full node into metadata (key _node_content) and rebuilds the TextNode on read. If you'd rather restore text from a custom metadata field (e.g. content), set text_field="content"; that field must exist in the metadata you write.

Q7: Dimension mismatch error The dimension you pass to create_index_from_bucket must exactly match your embedding model's output dimension (OpenAI text-embedding-3-small = 1536, bge-m3 = 1024, etc.).

Q8: What does clear() actually do? It calls delete_index and destroys the whole index, not just its data. After calling it you'll need to recreate the index via create_index_from_bucket.

---

License

Released under the MIT License.