Galmuri

English · 한국어

A Python framework for building services that collect, process, and serve documents.

Connect files, Google Drive documents, Slack messages, or your own data source. Choose how to transform and index the content, then make it available to other applications through MCP, HTTP, gRPC, CLI, or an interface you implement.

You develop your service in a generated project. Source connectors, processing functions, and output adapters are replaceable parts of that project. Galmuri executes the policies you configure and manages checkpoints, document updates, permissions, and retries.

Create your first project

Requires Python 3.12+. Install Galmuri from PyPI:

python3 -m venv .venv
. .venv/bin/activate
python -m pip install 'galmuri[mcp,grpc]==0.2.0'

Create a project, then create its first app:

python -m galmuri startproject example
cd example
python manage.py startapp sample

startproject creates the project settings and manage.py. startapp creates an app with its own source connection and sample documents. Register that app by editing config/settings.py:

APPS = ["apps.sample.app:sample"]

The generated app uses a local folder, the default parsers, local multilingual embeddings, and SQLite. Its first sync downloads model weights; no LLM API key is needed. Set EMBEDDING_CACHE before syncing to reuse an existing model cache.

python manage.py sync --app sample
python manage.py index-status --app sample
python manage.py call search --app sample --mode lexical --query welcome
python manage.py serve http

The search command returns the generated welcome.md document. Open http://127.0.0.1:8000/dashboard to browse the documents, then stop the server with Ctrl+C. Document processing and vector indexing finish separately; check index status for pending or failed work before using vector search.

For an example that does not download a model, follow Build your first document service.

Develop your app

The main files created by those two commands are:

example/
├── manage.py
├── config/
│   └── settings.py
└── apps/
    └── sample/
        ├── app.py
        ├── source/
        │   └── local.py
        ├── fixtures/
        │   ├── welcome.md
        │   └── clinic.md
        └── skills/
            └── usage.md

To connect your own folder, edit the path in apps/sample/source/local.py. To use another service, configure a provider from Duru or implement its source contract. Reading, writing, and service-specific API calls belong in the source provider.

apps/sample/app.py binds that source to the framework. Its minimal form is:

from galmuri import App
from .source.local import source

sample = App(name="sample", source=source)

Use the default processing to get started. When you need your own parser, transformation, or indexing method, write that implementation in the app and connect it here. The step-by-step tutorial shows the files to create and how to verify their output.

Each App owns one source connection and its processing, retrieval, and execution policies. To add another source, run python manage.py startapp manuals, configure its source, and add "apps.manuals.app:manuals" to APPS. Apps can use different processing, schedules, and stores within the same project.

What you can build

An output adapter—called an exporter in the integration guide—uses the shared Service to expose accepted documents and operations. Built-in adapters provide MCP, HTTP, gRPC, and CLI. A custom adapter implements run() and is registered in EXPOSURE_ADAPTERS. Automatic delivery and retries to arbitrary external destinations are outside that adapter contract.

For RAG, use Galmuri to develop the document and retrieval side of your system. Your existing LLM application consumes those results and owns prompts, answer generation, and conversation logic.

The older python -m galmuri new PATH command remains available as a shortcut that creates a project with a registered sample app. Its --process, --lexical, and --retrieval presets remain supported. The lexical and Qdrant presets use trusted host identities instead of the default account-enabled setup.

Customize processing

Inside your project, generate an app with editable processing functions:

python manage.py startapp articles --process

Register "apps.articles.app:articles" in APPS. The app includes:

apps/articles/
├── app.py
├── process.py
└── views/
    ├── __init__.py
    ├── normalization.py
    └── metadata.py

Edit views/normalization.py to change document text:

from dataclasses import replace

def normalize(documents):
    return tuple(
        replace(document, text=document.text.strip() + "\n")
        for document in documents
    )

In process.py, connect the function and the supplied metadata function:

from galmuri import Pipeline, Step
from .views.normalization import normalize
from .views.metadata import label

process = (
    Pipeline.default()
    .then(Step("normalize", process=normalize))
    .then(Step("label", process=label))
)

The generated app.py selects it with App(name="articles", source=source, process=process). Functions run when processing work executes. Add .then(Step(...)) for another step, or change declaration order to change the sequence.

Run the app and inspect its output:

python manage.py sync --app articles
python manage.py call search --app articles --mode lexical --query welcome

The returned document includes "pipeline": "articles" in its metadata.

These added steps receive and return document tuples. Default parsing runs first; document permission preparation follows your transformations. For a fully custom sequence, use Pipeline(*steps) with explicit file, documents, and terminal result boundaries. A plain App(process=my_function) accepts File and returns IngestResult. Default processing and Pipeline retain automatic retrieval when retrieval is omitted. A plain custom process/handler instead defaults to lexical-only; set retrieval=RetrievalConfig() explicitly to enable its embeddings and vector index. A plain function returning IngestResult also owns the returned grants; settings defaults are not added to that result.

Configure only the parts you need:

Config classes above are imported from galmuri, except SyncPolicy, which is in galmuri.collection.policies. For example, SyncPolicy(mode="scan", interval=86400) schedules a daily full scan. Omitted settings retain defaults.

Agent skill

The repository includes using-galmuri, an agent skill for creating apps, changing processing steps and checking document access. From this checkout, install it for Codex with:

mkdir -p "${CODEX_HOME:-$HOME/.codex}/skills"
cp -R skills/using-galmuri "${CODEX_HOME:-$HOME/.codex}/skills/"

Restart or reload skill discovery, then ask the agent to use $using-galmuri. Other compatible agents can install the same skill folder. Wheels also include it under share/galmuri/skills/using-galmuri in the Python installation's data directory.

Sources, models, and storage

Version 0.2.0 introduces indexed keyword search and structure-aware chunking. See search configuration, evaluation, and upgrade steps for Korean matching limits, SearchConfig, model input budgets, and migration.

Supply credentials and account/folder/channel IDs to the resource library. A provider with root_uri can be passed directly as App.source; otherwise use galmuri.sources.resources.Resources(provider, root_uri). Google-native documents require an explicit export format. Slack messages arrive as JSON resources; customize parsing to select the fields you need.

The current provider bridge scans snapshots and compares versions. It does not automatically subscribe to native Google Drive or Slack incremental feeds.

An App can set retrieval=RetrievalConfig(embedding=my_embedding, store=my_store, mode="hybrid"). Missing embedding/store providers are filled from defaults. The default vector store follows the App's selected database. EmbeddingConfig(process=..., query_process=..., spec=...) connects paired document-batch and query functions. Existing EmbeddingProvider, Chunker, and VectorStore implementations can also be passed directly.

Set retrieval=None to disable vector indexing. Selecting lexical as the default search mode alone does not disable vector builds. For PostgreSQL aliases/namespaces, Qdrant setup, and provider configuration, see the configuration and operations reference.

Accounts and document access

From a default or --process generated project:

python manage.py createsuperuser root
python manage.py serve http

The first command prompts for a password. Sign in at http://127.0.0.1:8000/dashboard to manage accounts, groups, document permissions, and personal MCP tokens. There is no default account or password.

Built-in ingestion and pipelines ending in document tuples resolve omitted IngestConfig grants from DEFAULT_PERMISSIONS in config/settings.py. Generated defaults allow reads and writes; explicit empty grants deny access. A custom process or terminal pipeline step returning IngestResult supplies its own document grants. Document's empty default grants do not become public by themselves. Source write support is still required, so the read-only sample exposes no write tool.

Group membership changes apply without reindexing. Ordinary administrators manage permissions; root can also bypass document ACLs. Token scopes, app restrictions, source capabilities, and version checks still apply to root.

For a read-only account token:

python manage.py createuser reader
python manage.py mcp-token reader --scope read --app sample --token-file .runtime/reader.token
FRAMEWORK_TOKEN_FILE=.runtime/reader.token python manage.py serve mcp

Local account commands require trusted OS access to the project. Restricted documents require verified credentials; tool arguments cannot assert a user/group. Invalid, expired, or revoked credentials are rejected.

Serve your app

Run from the generated project:

serve normally starts the scheduler and one local worker. Use --workers 0 when running workers separately. The MCP adapter is a stdio server. The integration tutorial includes stdio client configuration and a complete Python MCP client. HTTP deployment needs a public HTTPS origin and TLS proxy; generated loopback HTTP settings are for local use.

Build the supplied image from the framework checkout, with the sibling duru/ checkout present:

docker build --build-context duru=../duru -t galmuri-local .
docker run --rm galmuri-local --help

Mount a generated project at /workspace to retain its code and state.

Commands and recovery

Run from a generated project. Replace BUILD_ID with a build ID from index status. For management calls such as status, first create a root session after creating your root account: python manage.py login root --token-file .runtime/operator.token. It prompts for the password.

Standard Runtime workers persist accepted step outputs and resume after them. HTTP remote workers and custom workflows retry the whole process. Callbacks interrupted before their output is saved may run again: make external side effects idempotent.

Document publication and vector publication finish separately. Check index-status after sync; a drain can leave future retries or failed jobs. Change an implementation's revision when changing its behavior in place, then activate. State from older parser bindings may also require activate. See the reference for detailed recovery and store-cleanup commands.

Current limits

• Pipelines support linear steps; branches, loops, and DAG scheduling are not included.
• Google Drive/Slack providers currently support read/stat/list. Their account APIs are tested with local HTTP fixtures; live-account verification is separate.
• PDF/OCR parsers and OpenSearch require additional implementations.
• SQLite is for a single host. Use PostgreSQL to share metadata/account state across hosts, with compatible code and source settings in every process.
• SQLite/PostgreSQL vector stores use exact cosine search. Large-corpus throughput, ANN quality, and long-running production operation need separate evaluation.

Development

Clone the repositories next to each other for source development:

git clone https://github.com/imguno/duru.git duru
git clone https://github.com/imguno/galmuri.git galmuri
cd galmuri
python3 -m venv .venv
. .venv/bin/activate
python -m pip install -e ../duru -e '.[mcp,grpc,postgres,dev]'
python -m pytest -q
python -m ruff check src tests tools
python -m ruff format --check src tests tools
python -m build --wheel

Set FRAMEWORK_POSTGRES_DSN to a disposable PostgreSQL instance for remote DB tests. Optional live-model/LLM checks are separate from the ordinary suite; commands are in the reference.

Include a reproducible example and relevant checks with fixes or extensions. Keep the English and Korean READMEs aligned for user-facing changes.

See contributing, security reporting, and the release procedure.

License

Distributed under the BSD 3-Clause license.