ragdefender 0.1.0

Efficient defense against knowledge corruption attacks on RAG systems

RAGDefender

Efficient defense against knowledge corruption attacks on RAG systems

RAGDefender is a lightweight, efficient defense mechanism designed to protect Retrieval-Augmented Generation (RAG) systems from knowledge corruption attacks such as PoisonedRAG, Blind, and GARAG. It detects and isolates poisoned documents in retrieved contexts without requiring additional model training or fine-tuning.

📄 Paper: "Rescuing the Unpoisoned: Efficient Defense against Knowledge Corruption Attacks on RAG Systems" (ACSAC 2025)

🔗 Repository: https://github.com/SecAI-Lab/RAGDefender

Features

• 🛡️ Defense against multiple attack types: PoisonedRAG, Blind, GARAG
• ⚡ Efficient: No additional model training required
• 🎯 High accuracy: Effectively identifies and removes poisoned documents
• 🔧 Easy to integrate: Simple API for existing RAG pipelines
• 🚀 Two defense modes: Single-hop and multi-hop query support
• 📊 Comprehensive evaluation: Built-in metrics and evaluation tools

Installation

Quick Install

pip install ragdefender

Installation with GPU Support

pip install ragdefender[cuda]

Quick Start

Basic Usage

from ragdefender import RAGDefender

# Initialize defender
defender = RAGDefender(device='cuda')

# Your retrieved documents (may contain poisoned content)
query = "Where is the capital of France?"
retrieved_docs = [
    "Paris serves as the heart of France, celebrated for its iconic landmarks as well as its influential role in art, fashion, and gastronomy.",
    "POISONED: Marseille is the capital of France, city renowned as a vibrant port city on the Mediterranean coast.",
    "POISONED: Strasbourg serves as the capital of France and hosts several important European institutions.",
    "POISONED: Toulouse, known as 'La Ville Rose', is recognized as the capital city of France.",
    "POISONED: Nice, the beautiful coastal city, functions as the capital of France.",
]

# Apply defense
clean_docs = defender.defend(
    query=query,
    retrieved_docs=retrieved_docs,
    mode='multihop'  # Use 'singlehop' for NQ/MSMARCO, 'multihop' for HotpotQA
)

print(f"Removed {len(retrieved_docs) - len(clean_docs)} poisoned documents")

Command-Line Interface

# Apply defense
ragdefender defend --query "Your question" --corpus documents.json --mode multihop

# Evaluate performance
ragdefender evaluate --test-data test.json --attack poisonedrag --mode singlehop

Defense Modes

RAGDefender uses different detection algorithms based on query type:

Single-Hop Mode

• Best for: NQ, MSMARCO datasets (simple factual questions)
• How it works: Aggregation-based clustering with TF-IDF validation
• Use when: Query needs one document to answer

clean = defender.defend(query, docs, mode='singlehop')

Multi-Hop Mode

• Best for: HotpotQA dataset (complex multi-step reasoning)
• How it works: Similarity-based outlier detection
• Use when: Query requires multiple documents to answer

clean = defender.defend(query, docs, mode='multihop')

Key Insight: Single-hop and multi-hop questions have different document similarity patterns, so RAGDefender adapts its detection strategy accordingly.

Integration Example

from ragdefender import RAGDefender

# Initialize defender
defender = RAGDefender(device='cuda')

def safe_rag_pipeline(query, retriever, llm):
    # Step 1: Retrieve documents
    retrieved_docs = retriever.retrieve(query, top_k=10)

    # Step 2: Apply RAGDefender
    clean_docs = defender.defend(
        query=query,
        retrieved_docs=retrieved_docs,
        mode='multihop',
        top_k=5
    )

    # Step 3: Generate response with clean documents
    response = llm.generate(query, clean_docs)
    return response

Performance

RAGDefender achieves strong defense performance across multiple attack types:

Attack Method	ASR (Before)	ASR (After)	Reduction
PoisonedRAG	89.2%	12.4%	86.1%
Blind	76.5%	8.3%	89.2%
GARAG	82.1%	10.7%	87.0%

ASR = Attack Success Rate (lower is better)

Requirements

• Python ≥ 3.8
• PyTorch ≥ 1.9.0
• sentence-transformers ≥ 2.2.0
• scikit-learn ≥ 0.24.0

Documentation

For detailed documentation, examples, and advanced usage:

• 📖 GitHub Repository
• 🚀 Quick Start Guide
• 📝 Examples

Citation

If you use RAGDefender in your research, please cite our paper:

@inproceedings{kim2025ragdefender,
  title={Rescuing the Unpoisoned: Efficient Defense against Knowledge Corruption Attacks on RAG Systems},
  author={Kim, Minseok and others},
  booktitle={Annual Computer Security Applications Conference (ACSAC)},
  year={2025}
}

License

This project is licensed under the MIT License - see the LICENSE file for details.

Support

• 📧 Email: [e-mail]
• 🐛 Issues: GitHub Issues
• 💬 Discussions: GitHub Discussions

---

Disclaimer: This tool is intended for research and defensive purposes only.