viincci-rag 2.0.0

Universal multi-domain research system with RAG (Retrieval-Augmented Generation) capabilities

Viincci-RAG

Universal multi-domain research system with RAG (Retrieval-Augmented Generation) capabilities

🎓 Try It Now (Google Colab)

No installation required! Run these notebooks in your browser:

• Minimal Examples — Safe mock mode + real SerpAPI integration
• Complete Testing — All domains (poetry, medical, botany, art, carpentry)

🚀 Quick Start

Installation

# Install from source
pip install -e .

# With development dependencies
pip install -e ".[dev]"

# All features
pip install -e ".[all]"

Basic Usage

from viincci_rag import ConfigManager, RAGSystem, UniversalResearchSpider

# Initialize configuration
config = ConfigManager(domain="botany")

# Create RAG system
rag = RAGSystem(config)
rag.load_llm()

# Create research spider
spider = UniversalResearchSpider(config)

# Or import all classes
from viincci_rag import *

📦 What's Included

Component	Purpose
ConfigManager	Configuration management with domain support
RAGSystem	Retrieval-Augmented Generation pipeline
UniversalResearchSpider	Multi-domain research and web scraping
UniversalArticleGenerator	Content generation for any domain
SerpAPIMonitor	API credit monitoring and management
FloraDatabase	Database operations and management

🧪 Testing

# Run all tests
pytest tests/

# Run integration tests
pytest tests/test_integration.py -v

# With coverage report
pytest tests/ --cov=viincci_rag --cov-report=html

📚 Documentation

All documentation has been moved to the docs/ folder:

• docs/DOCS.md - Documentation index and quick reference
• docs/MIGRATION.md - Complete migration guide
• docs/RESTRUCTURING_SUMMARY.md - Architecture overview
• docs/BACKWARD_COMPATIBILITY_SHIMS.md - Technical details
• docs/package_structure.md - Packaging guide

🔄 Backward Compatibility

All old imports continue to work:

# Old import (still works)
from V4 import ConfigManager, RAGSystem

# New import (recommended)
from viincci_rag import ConfigManager, RAGSystem

# Both are identical

📋 Project Structure

viincci_rag/
├── core/              # Core RAG modules with wrappers
├── database/          # Database adapters
├── utils/             # Utility functions
├── config/            # Configuration files
└── templates/         # Output templates

V4/                    # Original codebase (unchanged)
docs/                  # Documentation
tests/                 # Test suite

🤝 Contributing

Contributions welcome! Please:

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Run tests: pytest tests/
5. Submit a Pull Request

📄 License

MIT License - See LICENSE file for details

📞 Support

• 📧 Issues: GitHub Issues
• 📖 Documentation: docs/
• 💬 Discussions: GitHub Discussions

---

Version: 4.0.0 | Status: Beta | License: MIT

## ✨ Features

- 🔬 **Multi-Domain Research**: Botany, medical, mathematics, carpentry, and more
- 🤖 **RAG System**: Retrieval-Augmented Generation for intelligent answers
- � **Multiple Database Backends**: SQLite, PostgreSQL, MongoDB, MySQL
- 🎯 **API Monitoring**: Built-in SerpAPI credit tracking
- ⚙️ **Fully Configurable**: Models, databases, content processing
- ✅ **Tested & Documented**: Comprehensive test suite and documentation
- 🔄 **Backward Compatible**: All old imports still work
# Clone the repository
git clone https://github.com/yourusername/viincci-rag.git
cd viincci-rag

# Install in development mode
pip install -e .

# Or install with all dependencies
pip install -e ".[all]"

For Development

# Install with dev dependencies
pip install -e ".[dev]"

# Run tests
pytest

# Or use the built-in test command
viincci-test

🔑 Setup

1. Get a SerpAPI Key: Sign up at serpapi.com

2. Set Environment Variable:

   export SERP_API_KEY='your_api_key_here'
   

3. Verify Installation:

   viincci-research --list-domains
   

📖 Usage

Command Line Interface

Basic Research Article

viincci-research -q "Rosa rubiginosa" -d botany

Plain Text Output

viincci-research -q "diabetes" -d medical --format text

JSON Structured Output

viincci-research -q "Pythagorean theorem" -d mathematics --format json

Arts & Humanities Research

viincci-research -q "Impressionism" -d art_history
viincci-research -q "Shakespeare sonnets" -d literature

Creative Writing with RAG

# Generate a poem
viincci-research -q "Van Gogh" -d art_history --content-type poem --rag

# Generate an essay
viincci-research -q "Baroque music" -d music --content-type essay --rag

Check API Status

viincci-research --check-credits

Python API

from V4 import ConfigManager, UniversalResearchSpider, RAGSystem, UniversalArticleGenerator

# Initialize configuration for a domain
config = ConfigManager(domain="mathematics", verbose=True)

# Perform research
spider = UniversalResearchSpider(config)
sources = spider.research("Pythagorean theorem")

# Generate article with RAG
rag = RAGSystem(config)
texts = [s['text'] for s in sources]
metadata = [s['metadata'] for s in sources]
rag.build_index(texts, metadata)
rag.load_llm()

generator = UniversalArticleGenerator(config, rag_system=rag)
article = generator.generate_full_article("Pythagorean theorem", sources)

# Save article
with open("article.html", "w") as f:
    f.write(article)

Domain Information

# List all domains
viincci-research --list-domains

# Get detailed domain info
viincci-research --domain-info medical

🏗️ Architecture

viincci-rag/
├── V4/                          # Main package
│   ├── __init__.py             # Package exports
│   ├── ConfigManager.py        # Configuration management
│   ├── Spider.py               # Web scraping & search
│   ├── RagSys.py              # RAG system implementation
│   ├── UniversalArticleGenerator.py  # Article generation
│   ├── ApiMonitor.py          # API credit monitoring
│   ├── FloraDatabase.py       # Database operations
│   ├── config/                # Configuration files
│   │   ├── domains.json       # Domain definitions
│   │   ├── ai_settings.json   # AI model settings
│   │   ├── api_monitor.json   # API monitoring config
│   │   └── ...
│   └── db/                    # Database directory
├── research_cli.py            # Command-line interface
├── test_v4.py                # Test suite
├── requirements.txt          # Dependencies
├── setup.py                  # Package setup
└── README.md                 # This file

🧪 Testing

Run the comprehensive test suite:

# Using pytest
pytest

# Using built-in test runner
viincci-test

# With verbose output
viincci-test --verbose

# Run specific test
pytest tests/test_config.py -v

📊 Configuration

All configuration is stored in V4/config/ as JSON files:

• domains.json: Define research domains, sources, questions
• ai_settings.json: LLM and embedding model settings
• api_monitor.json: API usage thresholds and alerts
• search_config.json: Web scraping parameters
• domain_reliability.json: Source reliability scores

Example: Add a New Domain

Edit V4/config/domains.json:

{
  "your_domain": {
    "name": "Your Domain Name",
    "description": "Description of your domain",
    "primary_sources": ["university", "research_institute"],
    "questions": [
      "what are the key concepts",
      "what are the applications"
    ],
    "keywords": ["keyword1", "keyword2"]
  }
}

🔧 Advanced Features

RAG System Customization

from V4 import ConfigManager, RAGSystem

config = ConfigManager()

# Change LLM model
config.set_llm_model("LiquidAI/LFM-40B-MoE")

# Initialize RAG with custom settings
rag = RAGSystem(config)
rag.load_llm(device="cuda", load_in_8bit=True)

# Query with custom parameters
result = rag.query(
    "What are the benefits?",
    k=10,
    max_new_tokens=500,
    temperature=0.8
)

API Cost Estimation

from V4 import SerpAPIMonitor, ConfigManager

config = ConfigManager()
monitor = SerpAPIMonitor(config)

# Estimate research cost
estimate = monitor.estimate_research_cost("Plant name", questions=4)
monitor.print_estimate(estimate)

# Check if can afford
if estimate['can_afford']:
    # Proceed with research
    pass

📚 Documentation

For detailed documentation, visit the Wiki.

Key Topics

• Installation Guide
• Configuration Reference
• API Documentation
• Domain Creation Guide
• RAG System Guide

🤝 Contributing

Contributions are welcome! Please read our Contributing Guidelines first.

1. Fork the repository
2. Create a feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add some AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• SerpAPI for search capabilities
• Hugging Face for transformers and models
• FAISS for efficient similarity search
• Wikimedia Commons for images

📞 Support

• 📧 Email: [e-mail]
• GitHub: MrViincciLeRoy
• 🐛 Issues: GitHub Issues
• 💬 Discussions: GitHub Discussions

🗺️ Roadmap

• Add more research domains
• Implement caching for search results
• Add web interface
• Support for more LLM providers
• Multilingual support
• Export to more formats (PDF, DOCX)
• Integration with reference managers

📈 Changelog

See CHANGELOG.md for version history.

---

Made with ❤️ by the Viincci-RAG Team