Osiris

Osiris is the project for osr, a small, data-oriented Lisp-to-Python compiler. The language is designed around readable Python output, explicit Python decorators, hygienic macros, static types, Unicode names, and tooling-readable metadata.

The current compiler implements a lossless tokenizer, a recoverable nom reader, surface AST lowering, hygienic macro expansion, name and alias resolution, typed HIR, defstruct, static schemas and records, and structured Python code generation. It also emits deterministic .osri interfaces and source maps, compiles source modules as one dependency graph, and validates locked static extension interfaces without importing Python packages.

The compiler-embedded standard library supplies Clojure-inspired control flow without making the reader or Rust core grow for every form. The implemented surface includes threading (->, ->>, cond->, some->, as->, doto), binding and branching (when, if-let, if-some, case, condp), comprehensions (for, doseq, dotimes), constant-stack loop/recur, letfn, defn- and trampoline, lazy sequences and reductions, structured exceptions/resources (assert, throw, time), typed ^:dynamic Var bindings, and the initial future/promise/locking primitives plus eager, ordered parallel forms (pmap, pcalls, pvalues) and the typed sequence predicates (empty?, seq?, coll?, sequential?). Dynamic values use Python contextvars, including context capture when a future is submitted. See the control-flow coverage matrix for exact semantics, tests, and intentionally deferred facilities such as with-bindings, with-local-vars, with-redefs, STM, Agents, and the complete Clojure sequence/transducer protocols. Osiris borrows these designs but does not claim Clojure compatibility.

Requirements

• Rust 1.85 or newer
• Python 3.11 or newer for Python packaging and extension builds
• uv for Python development

Project Quick Start

Create a new uv project with an Osiris source root and starter module:

osr init my-project
cd my-project
uv run osr run src/main.osr

To add Osiris to an existing uv project, run this from its root (or pass the directory explicitly):

osr init --existing
osr init --existing path/to/project

init preserves the existing pyproject.toml layout, comments, project metadata, and dependencies. It creates osiris.jsonc and src/main.osr only when those files do not exist, and asks uv to add osiris-lang to the development dependency group. Re-running the command is safe. A new project path must not already exist; use --existing when joining an established uv project.

Osiris discovers the nearest osiris.jsonc; the adjacent pyproject.toml continues to own Python package metadata and dependencies. JSONC comments and trailing commas are accepted. A typical configuration is deliberately small:

{
  "$schema": "https://raw.githubusercontent.com/mjason/osiris/main/schemas/osiris.schema.json",
  "source": ["examples"],
  "outDir": "dist",
  "targetPython": "3.11",
  "strict": true,
  "displayLocale": "zh-CN",
  "agent": {
    "model": "deepseek-v4-flash",
    "baseUrl": "https://api.deepseek.com/v1",
    "wireApi": "chatCompletions"
  }
}

source defines the complete project source scope. exclude contains project-root-relative glob rules shared by compilation and language tooling; a value without glob syntax, such as src/generated, also excludes its descendants. Patterns such as src/**/generated/** and src/**/*_test.osr can select files inside a source root when a project needs those rules. outDir is the default compile destination; artifact selection remains an explicit osr compile --emit option. One invocation targets one Python version. Changing targetPython invalidates target-sensitive analysis, interfaces, extension resolution, and build artifacts.

displayLocale accepts any well-formed BCP 47 language tag and is used by hover, completion, and signature help when Rich Metadata provides localized labels or documentation. For example:

• "zh-CN" requests Simplified Chinese labels and documentation.
• "en" requests English labels and documentation.
• "ja" requests Japanese labels and documentation.

It changes tooling presentation, not binding identity or generated Python. RFC 4647 lookup selects the closest authored locale and then falls back to the authored :default; the configuration is not a closed locale enum. An explicit locale sent by an LSP client takes precedence over the project value. osr init writes "displayLocale": "zh-CN" by default. LSC intentionally does not inherit it: pass --locale <bcp47> when a finite CLI query needs a particular language.

osr lsa "<request>" is the finite Language Server Agent for explaining Osiris and generating compiler-validated examples. Examples are compiled as temporary entries in the current workspace and executed with the project Python; ordinary imports and ~python use the same staging rules as osr run. Only the captured runtime value is reported as result. It returns JSON by default and includes a sessionId; pass that ID with --session for a follow-up. The API key is read from OSR_API_KEY in the environment or project .env. OSR_MODEL, OSR_BASE_URL, and OSR_WIRE_API override the agent object. The default chatCompletions calls /chat/completions; set wireApi or OSR_WIRE_API to responses for /responses. Protocol fallback is never implicit. Use --file <path> to explicitly include one local source file as context. Use --at <path>:<line>:<column> when a question is anchored to one expression. For broader feature requests, LSA performs bounded, read-only searches over a libSQL semantic graph and follows with precise LSP queries before generating an example. Compiler-owned languageService evidence keeps definitions, signatures, references, ambiguity, and source locations separate from model interpretation. Only selected Osiris forms use provider context; the complete workspace and raw Python implementation are never uploaded. The disposable graph is stored at .osiris/cache/language-graph.sqlite3, while session JSONC is kept under .osiris/cache/agent/; neither enters dist. Graph-only searches open a matching cache before workspace analysis. Source, configuration, lock, or static-interface changes refresh it automatically; unchanged files reuse hashes from a persistent input manifest, so hot validation does not reread the workspace. osr lsc cache rebuild remains available as a full recovery rebuild.

With that configuration and examples/hello.osr:

cargo run --bin osr -- check examples/hello.osr
cargo run --bin osr -- build
cargo run --bin osr -- watch

The multi-file examples/tutorial/app.osr demonstrates importing another Osiris module with :as and :refer, importing a macro with import-for-syntax, and keeping Python py/import separate:

(import tutorial.transforms :as transforms :refer [sum-values])
(import-for-syntax tutorial.macros :refer [unless])
(py/import math :as math)

Run cargo run --bin osr -- check examples/tutorial/app.osr to analyze the whole local dependency graph. See examples/README.md for the module-to-path mapping and generated outputs.

check parses and validates the project and leaves the working tree unchanged. build compiles the complete project described by osiris.jsonc, prints the output directory (dist/ by default), and publishes one artifact set atomically. watch reruns that same build when a non-excluded .osr source changes. compile remains the lower-level command for explicit source and --emit control.

Successful project builds keep one bounded cache entry in .osiris/cache/. It is separate from dist/, is never published, and can always be deleted. Unchanged builds leave an identical dist/ untouched; a missing or stale dist/ can be restored from the validated cache. Generated Python is formatted by the Ruff formatter embedded in osr before source maps are produced, so no external ruff command or project Ruff configuration is required. Authored fallback documentation becomes Python docstrings, and compiler-owned names are kept recognizable without exposing hygienic internal identities.

• dist/hello.py is the readable generated Python module.
• dist/hello.osri is the public, versioned Osiris compilation interface used by downstream modules and tools.
• dist/hello.py.map maps generated Python spans back to source and macro-expansion spans.
• A distribution-level *.records.json sidecar is emitted only when the compiled modules own public static records (or when --emit records is requested).

Python dependencies and Osiris extensions are ordinary Python project dependencies. Add them from PyPI (or another index/path supported by uv) in [project].dependencies, then let uv resolve and lock them. The compiler automatically reads osiris.toml and .osri resources only from distributions reachable in the runtime lock graph; it never imports extension Python code or scans unrelated installed packages during discovery.

Publishing a Package

An Osiris package is an ordinary Python distribution whose wheel contains compiled .osri interfaces and an automatically generated dist-info/osiris.toml marker. Create one with:

osr init --package acme-osiris
cd acme-osiris
uv lock
uv build --python 3.11
uv publish dist/*

The generated pyproject.toml pins the installed compiler distribution and selects its bundled PEP 517 backend:

[build-system]
requires = ["osiris-lang==<osr-version>"]
build-backend = "osiris_build"

osr init --package acme-osiris creates src/acme_osiris/core.osr with module acme_osiris.core. Each public module is compiled into readable Python plus an .osri interface; the backend adds one [[extension]] marker entry for each interface, using the module name (acme_osiris.core) as its ID. Do not write osiris.toml by hand.

To convert an existing uv package, run osr init --existing --package from its root. The command preserves existing metadata and refuses to replace a different build backend. If that package needs Hatchling, maturin, or another backend for additional native build work, backend composition is not yet supported and should be handled as a separate distribution.

Consumers install the published extension exactly like any other dependency:

uv add acme-osiris
uv lock
uv run osr check src/main.osr

The compiler follows the consumer's locked runtime dependency graph and reads the extension's static marker and interfaces without importing its Python package during discovery. Public interface dependencies of an extension must therefore be declared in [project].dependencies, so they are preserved as standard Requires-Dist metadata.

Native CLI

cargo run --bin osr -- --version
cargo run --bin osr -- check source.osr
cargo run --bin osr -- build
cargo run --bin osr -- watch
cargo run --bin osr -- compile source.osr
cargo run --bin osr -- expand source.osr
cargo run --bin osr -- fmt --check source.osr
cargo run --bin osr -- lsc semantic source.osr --format json
cargo run --bin osr -- lsc hover osiris.core/map --locale en
cargo run --bin osr -- lsc workspace-search "format message" --format json
cargo run --bin osr -- lsc symbol-context --at examples/hello.osr:10:3 --format json
cargo run --bin osr -- syntax
cargo run --bin osr -- doc '{ documentationCapabilities { snapshotId } }'
cargo run --bin osr -- lsp
cargo test --all-targets --all-features

check runs the frontend and semantic gates. build emits readable Python, an .osri compilation interface, and a .py.map source map into dist/ by default. expand shows macro output, fmt applies the canonical formatter, and lsc exposes the finite semantic and navigation operations used by the LSP without requiring an editor protocol. syntax prints the embedded English language manual, while doc queries the embedded read-only documentation snapshot with GraphQL. Compilation errors return status 1; command-line misuse returns status 2.

The reader is implemented as composable nom grammar productions over a lossless token stream. All whitespace, commas, comments, original Unicode spelling, and raw string spelling remain available to future formatting and LSP stages. Symbols and keywords also carry an NFC canonical spelling for collision-safe name resolution.

Python package

The PyPI wheel carries osr as a native Rust executable. Python and its packaging tools install the wheel, but they do not launch or host the CLI. The same wheel contains the osiris_build PEP 517 backend used by Osiris source distributions. It does not provide a shared runtime package: each build links only reachable standard support into the generated distribution's private __osiris_runtime__ package, so deployed Python does not depend on osiris-lang. Python dependencies continue to be declared in pyproject.toml and locked by uv.

The PyPI distribution is named osiris-lang because the osiris project name is already occupied. Its importable build-backend package is osiris_build:

uv tool install osiris-lang
osr --version

For repository development:

uv sync
uv run osr --version

The package version is defined once in Cargo.toml; maturin uses it for the platform wheel and places the native executable in the wheel's scripts area. Consequently osr, osr watch, and osr lsp run without a Python interpreter process.

VS Code

The extension lives in editors/vscode and delegates all semantic behavior to osr lsp. Until Marketplace publishing is enabled, open the repository's GitHub Releases, select the latest vscode-vX.Y.Z release, download its .vsix, and run Extensions: Install from VSIX... in VS Code.

Maintainers publish Python releases with a vX.Y.Z tag and VS Code releases with a separate vscode-vX.Y.Z tag. Both tags must match the corresponding package version committed in the repository. Trusted Publisher fields and the full tag procedure are documented in docs/releasing.md.

The current language design is in docs/language-design.md. Compiler ownership and the kernel/macro/extension boundary are documented in docs/architecture.md.