ecdallm

ecdallm is a lightweight Retrieval-Augmented Generation (RAG) application that lets you chat with your own documents using either a local LLM or an external OpenAI-compatible provider.

It combines:

• FastAPI web interface
• Local embedding pipeline (FastEmbed)
• Persistent vector storage (ChromaDB)
• Document ingestion (PDF, TXT, DOCX)
• CLI launcher
• Local LLM support (e.g., LM Studio)
• External LLM support (OpenAI-compatible APIs)

The goal is to provide a simple, reproducible environment for document-grounded LLM interaction with flexible model connectivity.

Overview

ecdallm allows you to:

1. Upload documents
2. Index them into a vector database
3. Run semantic retrieval
4. Query an LLM with grounded context

All embeddings and vector storage run locally.

The chat model can run:

• locally (LM Studio, Ollama, etc.)
• externally (OpenRouter or OpenAI-compatible APIs)

This makes the system suitable for:

• research environments
• private document analysis
• offline experimentation
• RAG prototyping
• hybrid local/cloud workflows

Prerequisites

Make sure Python 3 is installed.

On some systems (especially macOS installations from python.org), the commands are named python3 and pip3 instead of python and pip.

You can check which commands are available:

python --version
python3 --version

pip --version
pip3 --version

If python or pip returns:

command not found

use the python3 / pip3 variants instead.

Typical macOS setup:

python3 --version
pip3 --version

ecdallm can support both the /v1/chat/completions and vLLM, /v1/completions, endpoints. It can now automatically detect the correct endpoint and switch between them when needed.

Installation

Install from PyPI:

pip install ecdallm

If your system uses pip3:

pip3 install ecdallm

Running the application

Start the CLI:

ecdallm

If the command is not found, try:

python3 -m ecdallm

or ensure your Python scripts directory is in your PATH.

The CLI will:

• find a free port (starting from 8000)
• start the FastAPI server
• open the browser automatically

Example output:

ecdallm running at http://127.0.0.1:8000/
INFO: Uvicorn running on http://127.0.0.1:8000

LLM Configuration

When the application starts, click Continue and choose:

• Local LLM
• External LLM

Configuration is stored in the browser session.

Embeddings always run locally using FastEmbed with:

nomic-embed-text-v1.5

Using a local LLM

ecdallm expects an OpenAI-compatible endpoint.

For example, with LM Studio:

1. Start LM Studio server
2. Load a chat model
3. Enable the local API server

Typical endpoint:

http://localhost:1234/v1

Default configuration:

Base URL: http://localhost:1234/v1
API Key: lm-studio

The backend automatically detects the available chat model via:

GET /models

Using an external LLM

ecdallm can connect to any OpenAI-compatible API provider.

Examples include:

• OpenRouter
• OpenAI-compatible gateways
• Self-hosted inference APIs

Example configuration (OpenRouter):

Base URL: https://openrouter.ai/api/v1
Model: openrouter/aurora-alpha
API Key: sk-or-...

Steps:

1. Create an account with the provider
2. Generate an API key
3. Choose a chat model
4. Enter the configuration in the web interface

When validating, ecdallm:

• checks connectivity
• performs a test chat completion
• stores configuration in session storage

Your API key is sent only to your backend for validation and is not used directly in the browser.

Embeddings remain local.

Supported document types

• PDF
• TXT
• DOCX

Workflow

1. Upload documents

Use the Upload page to add files.

2. Index documents

Files are automatically indexed into ChromaDB using FastEmbed.

3. Chat with documents

Open the Chat page and ask questions.

The assistant will:

• retrieve relevant chunks
• build a grounded prompt
• query the configured LLM
• return a concise answer

Project structure

ecdallm/
├── cli.py
└── app/
    ├── main.py
    ├── rag.py
    ├── paths.py
    ├── search_engine.py
    ├── vector.py
    ├── templates/
    ├── static/
    ├── uploads/
    └── rag_store/

Notes

Embeddings and retrieval always run locally.

The chat model can be:

• local (LM Studio, Ollama, etc.)
• external (OpenAI-compatible providers)

This keeps the system flexible while maintaining local document processing.

Erasmus Data Collaboratory

Developed by the Erasmus Data Collaboratory (ECDA).

• Zaman Ziabakhshganji --- creator and maintainer
• Farshad Radman --- co-author and contributor
• Jos van Dongen --- co-author and contributor

License

MIT License