Skip to main content

modularRAGagain

基于 LangChain + 阿里云 DashScope 的模块化 RAG(检索增强生成)流水线,把检索链路拆成 11 个可组合模块,按需拼装即可。

特性

# 模块 说明
1 查询规范化 去多余空白、统一格式
2 查询改写 将多轮/口语化问题改写为独立检索查询
3 HyDE 生成假设性答案做语义检索
4 向量检索 Chroma + DashScope Embeddings
5 BM25 检索 jieba 中文分词 + BM25
6 混合检索 EnsembleRetriever + RRF 融合
7 结果去重 按正文去重
8 重排序 DashScope qwen3-rerank 精排
9 相似度过滤 余弦相似度阈值过滤并写回 metadata
10 上下文压缩 LLMChainExtractor 抽取式压缩
11 长上下文重排 LongContextReorder 防"迷失在中间"

安装

pip install modularragagain

需要环境变量:

set DASHSCOPE_API_KEY=sk-xxxx        # Windows
export DASHSCOPE_API_KEY=sk-xxxx     # Linux / macOS

快速开始

import os
from modularragagain import ModularRAG

rag = ModularRAG({
    "persist_directory": "./chroma_db",   # 需已存在的 Chroma 知识库
    "collection_name": "study",
    "embedding_model": "text-embedding-v4",
})

question = "什么是 RAG?"
query = rag.normalize_query(question)
query = rag.rewrite_query(query)
hyde = rag.generate_hyde(query)

docs = rag.vector_search(hyde, rag.db, top_k=5)
docs = rag.deduplicate(docs)
docs = rag.rerank_documents(query, docs, top_k=3)
docs = rag.similarity_filter(query, docs, rag.embeddings, threshold=0.4)
docs = rag.compress_context(query, docs)
docs = rag.reorder_documents(docs)

for d in docs:
    print(d.metadata.get("cosine_similarity"), d.page_content[:80])

完整可运行示例见 examples/demo.py,它会在本地自建一个测试知识库并跑通全部 11 个模块。

依赖

langchain-core / langchain-openai / langchain-community / langchain-chroma / langchain-classic / chromadb / dashscope / jieba / python-dotenv

License

未指定(如需开源请补充 LICENSE)。

Metadata

Release files for modularragagain 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for modularragagain 0.1.0
File Size Uploaded Provenance Actions
modularragagain-0.1.0.tar.gz 8.0 kB Without provenance Details for modularragagain-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for modularragagain 0.1.0
File Interpreter ABI Platform Actions
modularragagain-0.1.0-py3-none-any.whl Python 3 none any Details for modularragagain-0.1.0-py3-none-any.whl

Total release size: 13.5 kB

Release files / modularragagain-0.1.0.tar.gz

Download URL modularragagain-0.1.0.tar.gz
Size 8.0 kB
Tags Source
SHA-256 checksum
How to use checksums
42d73133a51414d7b033b068957fa8c00974d201bef0d461eff1929b5f7bda82
BLAKE2b-256 checksum
How to use checksums
9d67ab560035a0becc1bece6fdd6d2409cd76926cc8dbda5ea97187da39883c6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.5

Release files / modularragagain-0.1.0-py3-none-any.whl

Download URL modularragagain-0.1.0-py3-none-any.whl
Size 5.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
e399b9bd4329705f261e0bdde433e880dc8a8d3fdbab2ebc3added0534fcc60c
BLAKE2b-256 checksum
How to use checksums
94b8e4c16ee5d410cb6eaf6766ee1187356b92fee83f69c2c411f024670b545f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.5

Security for modularragagain 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page