Ershu Worker

简体中文

Ershu Worker is the background analysis service for Ershu. It is for operators running an Ershu installation and developers working on repository analysis. The worker claims jobs from the shared queue, inspects source repositories through read-only tools, and writes code indexes, project relationships, documents, and search data back to Ershu.

This package is one part of Ershu, not a standalone repository scanner. It does not expose an HTTP or MCP endpoint and it does not create analysis jobs. Use the Ershu deployment repository when you need the complete system.

What the worker does

A run can include the following work, depending on its type and the state of the repository:

• prepare a remote repository cache or an approved local workspace;
• identify source files, manifests, languages, symbols, calls, imports, and public entry points;
• build repository and project knowledge relationships;
• extract OpenAPI, GraphQL, Protocol Buffers, Thrift, AsyncAPI, generated client, database schema, message channel, and framework route facts;
• generate, review, and publish project documents with source references;
• update keyword and vector search data;
• record progress, warnings, model usage, and a final run snapshot.

A deterministic refresh only updates repository facts, code indexes, search indexes, and knowledge relationships. It does not rewrite AI-generated documents as a side effect.

Place in Ershu

Frontend -> API -> PostgreSQL queue -> Worker -> PostgreSQL
                                      |       -> Milvus
                                      |
                                      -> read-only repository workspace

MCP reads the project facts and documents stored by the Worker.

Each Ershu component has a separate job:

• ershu-core owns shared models, database services, settings contracts, and provider clients.
• The API owns HTTP, authentication, and job creation.
• The Worker owns queue consumption, repository analysis, document workflows, code intelligence, and LSP orchestration.
• MCP exposes stored results and refresh operations to clients.
• The deployment repository selects compatible component, package, and container versions.

The Worker imports ershu-core. It does not import the API or MCP packages.

Safety boundary

Repository analysis inspects project files without running the project:

• analysis tools expose bounded repository, symbol, index, graph, and document operations instead of a general shell;
• LSP containers mount the repository read-only, run without a network, drop Linux capabilities, and have CPU, memory, process, session, and time limits;
• strict LSP mode disables or limits automatic dependency loading and project behavior where the language server supports it;
• optional dependency preparation runs separately, uses supported dependency manifests, and does not share its network access with the LSP container;
• repository credentials, package source credentials, model keys, and ENCRYPTION_KEY belong in deployment configuration, not in source control or generated documents.

Semantic LSP mode loads more project context than strict mode. Keep it disabled for repositories you do not trust.

Requirements

• Python 3.14 or later
• uv
• a compatible ershu-core package and database schema
• PostgreSQL for the queue, run state, documents, and graph data
• Milvus for vector search
• configured model and embedding channels for runs that need them
• Docker only when LSP analysis or managed dependency preparation is enabled

Install

Install the Worker as part of a tested component set through the ershu installer package:

pip install "ershu[worker]"

To keep the Worker command in its own UV-managed environment, install the component directly:

uv tool install "ershu-worker==1.0.0.post20260812"

For development from a checkout:

uv sync --extra dev --locked

The locked development environment must be able to resolve the ershu-core version recorded in uv.lock.

Configure and start

Copy .env.example to .env, then replace its development values. The example file is the reference for environment settings used by the Worker process.

The main groups are:

At minimum, use a current database schema, a reachable database and Milvus instance, a strong ENCRYPTION_KEY, and a writable persistent REPOSITORY_STORAGE_PATH. Configure repository credentials and model channels through the API or frontend before queuing a run. Do not commit .env.

Start from the source checkout with:

uv run ershu-worker

If installed with uv tool install, run:

ershu-worker

The process validates its settings and analysis registries before it claims a job. It also initializes the structured parsers under REPOSITORY_STORAGE_PATH and runs a parser self-check. A healthy startup includes these log messages:

analysis worker started
structured parser self-check passed for configured languages

The Worker does not listen on a port. Monitor the process, startup self-check, queue claims, run heartbeats, warnings, and final run states instead of probing an HTTP health route.

LSP images

LSP is off by default. This repository contains six runtime images because their commands and safety policy are coupled to the Worker:

Build the default local tags from the repository root:

docker build -t ershu-lsp-python:basedpyright docker/lsp/python
docker build -t ershu-lsp-typescript:typescript-language-server docker/lsp/typescript
docker build -t ershu-lsp-go:gopls docker/lsp/go
docker build -t ershu-lsp-rust:rust-analyzer docker/lsp/rust
docker build -t ershu-lsp-clangd:clangd docker/lsp/clangd
docker build -t ershu-lsp-java:jdtls docker/lsp/java

Enable only the languages you need:

REPOSITORY_LSP_ENABLED=true
REPOSITORY_LSP_LANGUAGES=["python","typescript"]
REPOSITORY_LSP_DEFAULT_MODE=strict
REPOSITORY_LSP_ALLOW_SEMANTIC_MODE=false

LSP_IMAGES maps each adapter to the image tag used by the deployment. Image Dockerfiles and image release jobs belong with this component; the deployment repository should pin published image tags and supply the mapping.

For v1.0.0-20260812, the release workflow publishes versioned GHCR images such as ghcr.io/ershu-lab/ershu-lsp-python:v1.0.0-20260812. The deployment repository should use the exact tag instead of latest. Published images currently target linux/amd64.

Managed dependency preparation

LSP stays offline. If a language server needs installed dependency metadata, the Worker can run a separate short-lived download container before starting it:

REPOSITORY_DEPENDENCY_RESOLUTION_MODE=managed
REPOSITORY_DEPENDENCY_SOURCE_POLICY=configured_or_public

Managed preparation currently recognizes:

pnpm and Yarn lock files are not prepared. Java dependency preparation is disabled because Maven and Gradle project loading can execute repository-controlled extensions or build scripts. C and C++ have no managed package source in this Worker.

The source policy accepts configured sources, public sources, or configured sources with public fallback. HTTP package sources remain blocked unless the deployment explicitly permits them. A host-specific CA bundle and download cache can be supplied through the matching settings in .env.example.

Shutdown and recovery

On SIGTERM or an interactive stop, the Worker stops claiming jobs, drains background search work, shuts down LSP sessions, and returns its active runs to the queue. If the process exits before that handoff completes, another Worker can recover the run after its queue lease expires.

Transient database and infrastructure failures are retried. Programming and configuration errors stop the process so an operator can see the failure instead of leaving a broken Worker in the pool.

Develop and test

The main source areas are:

Run the repository checks with uv:

uv run pytest -q
uv run ruff check src tests scripts
uv run ruff format --check src tests scripts
uv build --no-sources --out-dir dist
uv run --no-project scripts/check_package_contents.py dist

Tests marked integration need the external service named by the test, such as PostgreSQL, Milvus, or Docker. The ordinary test suite must not require API or MCP source code.

License

Ershu Worker is licensed under the Apache License 2.0.