sigil-lang 0.1.0

A programming language for LLM-to-LLM communication that transpiles to Python

SIGIL

The programming language machines deserve.

91% fewer output tokens on real apps · 86% faster generation · 85% cheaper

Website · Playground · Discord · Spec

---

Status: Alpha — compiler, CLI, and VS Code extension functional. API may change.

What is Sigil?

Sigil is a programming language designed for LLM-to-LLM communication. It is not meant for humans to read or write. It is the language AI agents use when generating, sharing, and executing code — optimised for token density, not human readability.

Every mainstream language was designed for humans. That made sense when humans wrote and read code. It doesn't make sense when machines are doing both. Sigil strips out everything machines don't need — verbose keywords, whitespace formatting, comments, syntactic sugar — and keeps pure computational intent.

Python (14 tokens):
def calculate_average(numbers: list[float]) -> float:
    """Calculate the arithmetic mean."""
    return sum(numbers) / len(numbers)

Sigil (5 tokens):
@avg [f]>f = +$/# $

Sigil transpiles to Python, Rust, Go, or TypeScript for execution. Think of it as a hyper-efficient IR (intermediate representation) for AI-generated code.

Measured Token Reduction

Complete e-commerce app (product catalog, cart, checkout, dark theme UI) generated in one shot:

Metric	Python	Sigil	Savings
Output tokens	6,677	617	91% fewer
Lines of code	695	30	96% fewer
Generation time	66.4s	9.4s	86% faster
Cost	$0.10	$0.015	85% cheaper

The app runs on localhost with search, cart, checkout, and order confirmation.

Functional tasks (23 tasks, 100% accuracy):

Strategy	Python tokens	Sigil tokens	Reduction
Direct	1,037	474	54%
Dense (compact prompt)	1,037	416	60%
Compact (post-processed)	1,037	418	60%

Why?

• Token tax: Every token spent on human-readability features is a token an LLM doesn't need. At scale (millions of agent interactions/day), this is a real cost.
• Context windows: Density means more working space in finite context windows, where attention degrades with length.
• Capability, not just cost: Even with cheap inference, fitting more program state into context enables capabilities that raw token savings alone do not.

Architecture

LLM reasons about task
    -> generates Sigil
        -> Sigil transpiles to target language
            -> executes
                -> structured output returns to LLM

Current target: Sigil -> Python (via Python ast module)

Future targets: Sigil -> Rust, Go, TypeScript, WASM, bytecode

Project Structure

sigil/
├── grammar/            # Tree-sitter grammar definition
│   └── grammar.js      # Formal BNF-equivalent grammar
├── transpiler/         # Sigil -> Python transpiler
│   ├── parser.py       # Tokeniser + parser (AST generation)
│   ├── ir.py           # Intermediate representation
│   ├── codegen.py      # Python code generation from IR
│   └── cli.py          # Command-line interface
├── benchmarks/         # Token economy benchmarks
│   ├── humaneval/      # HumanEval benchmark translations
│   ├── mbpp/           # MBPP benchmark translations
│   └── results/        # Benchmark results and analysis
├── validation/         # LLM generation accuracy tests
│   ├── test_generate.py    # Can LLMs generate valid Sigil?
│   ├── test_roundtrip.py   # Sigil -> Python -> Sigil fidelity
│   └── test_accuracy.py    # Comparison vs Python generation
├── spec/               # Language specification
│   └── sigil-spec.md   # Formal language spec (BNF + semantics)
├── examples/           # Example Sigil programs
└── DESIGN_LOG.md       # Dated design decisions

Installation

pip install sigil-lang

For development (includes pytest and black):

pip install sigil-lang[dev]

For LLM API integration (includes anthropic and openai SDKs):

pip install sigil-lang[api]

Quick Start

# Transpile a .sigil file to Python
sigil transpile examples/hello.sigil --target python

# Transpile and execute
sigil run examples/hello.sigil

# Parse and type-check without executing
sigil check examples/hello.sigil

# Format a .sigil file
sigil fmt examples/hello.sigil

# Count tokens (compare Sigil vs generated Python)
sigil tokens examples/hello.sigil --compare

From Source

git clone [e-mail]:eduardragea/sigil.git
cd sigil

python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

# Run tests
python -m pytest tests/

Gradual Adoption: Inline Sigil in Python

You don't need to rewrite your codebase. Embed Sigil snippets directly in existing Python files and migrate function-by-function.

from sigil.inline import transpile, sigil_func

# Option 1: transpile a Sigil string and exec it
exec(transpile("@avg l = +l/#l"))
print(avg([1, 2, 3]))  # 2.0

# Option 2: replace a function stub with a Sigil implementation
@sigil_func("@greet n:s>s = f'Hello {n}'")
def greet(n): ...

print(greet("world"))  # Hello world

Both approaches produce standard Python at import time -- no runtime overhead and full compatibility with type checkers and debuggers.

Language Reference (Draft)

Concept	Python	Sigil
Variable binding	x = 42	x:42
Function definition	def f(x): return x+1	@f x = x+1
Conditional	if x > 0: a else: b	x>0 ? a : b
List comprehension	[x**2 for x in xs if x>0]	xs|>0|^2
Map + Filter	map(fn, filter(p, xs))	xs|p|>fn
Error handling	try: ... except: ...	!{...} ~> {...}
Type annotation	x: int = 5	x:i=5
Module import	from math import sqrt	<math.sqrt>
Async operation	await fetch(url)	~>fetch url
Pattern match	match x: case 1: ...	x~{1:... _:...}

Docker

# Build the image
docker build -t sigil-lang .

# Run Sigil CLI via Docker
docker run --rm -v $(pwd):/work sigil-lang transpile /work/app.sigil

# Run playground (API + web)
docker compose up

The playground exposes the API on http://localhost:8000 and the web UI on http://localhost:3000.

Roadmap

Done

• Formal grammar (43+ rules, Tree-sitter parser)
• Sigil → Python transpiler with type inference
• CLI (transpile, run, check, fmt, tokens, init)
• VS Code extension, playground web app, REPL
• AI tool integrations (Claude Code MCP, Cursor, Copilot, Codex, Grok)
• JS/TS/Go transpilation targets, 100 codebook templates, diff protocol

Phase 1: Trust the Transpiler ✅

• 100% e2e accuracy — 23/23 functional tasks pass with zero fine-tuning
• Codegen failures 35 → 13 — pipe map, fold lambda, filter, imports fixed
• Python-style calls — range(1, n+1), s.replace(" ", "") work natively
• Raw Python escape — %%{complex_python}%% for patterns Sigil can't express
• Negative slices — xs[::-1] works correctly

Phase 2: Break the 60-80% Barrier

Current: 29% on apps (47% of Sigil is %%{...}%% escape hatches). Theoretical: 63% without escapes.

• Fix dict literal parsing — json.dumps({"key": v}) in all positions (#281)
• Multi-function file parsing — external scanner for body termination (#284)
• Template system — REST_CRUD "tasks" Task {fields} → 50 lines Python (#282)
• Full-stack app — frontend + backend both in Sigil (#283)

Phase 3: Launch

• Publish v0.1.0 to PyPI (#147)
• Publish VS Code extension (#182)
• Write launch post (#149)

Phase 4: Sigil as a Reasoning Language

• Sigil-Plan — agent planning notation (#216)
• Sigil Agent — coding agent that thinks in Sigil (#218)
• Sigil-Think — compressed chain-of-thought (#217)

Phase 5: BaseLLM — Token-Native Encoding

Every number system is optimized for its consumer. Base2 for circuits. Base10 for humans. BaseLLM for transformers.

• Layer 1: Custom tokenizer — Sigil operators as single tokens, +30-50% compression (#293)
• Layer 2: Semantic code tokens — one token = one programming pattern (#294)
• Layer 3: Semantic reasoning tokens — one token = one reasoning primitive (#295)
• Target: 80-90% fewer tokens for complete agent sessions (code + reasoning + planning)

AI Tool Integrations

Sigil works with every major AI coding tool:

Tool	Setup	What it does
Claude Code	MCP server + skills	4 tools: parse, transpile, validate, count tokens
Cursor	.cursor/rules/sigil.mdc	AI generates correct Sigil in .sigil files
GitHub Copilot	.github/copilot-instructions.md	Copilot Chat understands Sigil syntax
OpenAI Codex	AGENTS.md	Codex reads project context automatically
Grok	FastAPI tool server	Function calling for parse/transpile/tokens

See integrations/ for setup instructions.

Constrained Decoding (GBNF)

Force LLMs to generate only valid Sigil syntax using grammar-constrained decoding. This eliminates syntax errors entirely -- the model cannot produce tokens that violate the grammar.

Two grammar files are provided in spec/:

File	Format	Used by
spec/sigil.gbnf	GBNF (GGML BNF)	llama.cpp, llama-cpp-python, text-generation-webui
spec/sigil.ebnf	Standard EBNF (ISO 14977)	Outlines, guidance, lm-format-enforcer

llama.cpp

Pass the grammar file with --grammar-file:

llama-cli \
  -m model.gguf \
  -p "Write a Sigil function that reverses a list:" \
  --grammar-file spec/sigil.gbnf

Or load the grammar string in llama-cpp-python:

from llama_cpp import Llama

llm = Llama(model_path="model.gguf")

with open("spec/sigil.gbnf") as f:
    grammar_text = f.read()

output = llm(
    "Write a Sigil function that reverses a list:",
    grammar=LlamaGrammar.from_string(grammar_text),
)

Outlines (vLLM / transformers)

Outlines accepts EBNF grammars for structured generation:

import outlines

model = outlines.models.transformers("model-name")
generator = outlines.generate.cfg(model, open("spec/sigil.ebnf").read())

result = generator("Write a Sigil function that computes factorial:")

guidance

from guidance import models, gen

lm = models.LlamaCpp("model.gguf")

with open("spec/sigil.gbnf") as f:
    grammar = f.read()

lm += "Generate Sigil:\n" + gen(name="code", grammar=grammar)

The Three-Layer Vision

Sigil is evolving from code compression into a full reasoning language for AI agents:

Layer 1: Sigil-Code   — compress Python syntax          (46% savings)  ← SHIPPED
Layer 2: Sigil-Plan   — compress agent planning          (70-80% est.) ← NEXT
Layer 3: Sigil-Think  — compress chain-of-thought        (70-80% est.) ← RESEARCH

Why this matters: An agent that thinks in Sigil-Plan and generates Sigil-Code has 2x the effective context window of a Python-native agent at the same API cost. This isn't just cheaper — it enables capabilities that don't fit in a Python agent's context.

Sigil-Plan example:
  P:add-error-handling target:f
    R:file → S:find @f → C:has !{}~>{}
    ?!C → E:wrap !{body}~>{fallback} → T:pytest → G:commit

vs natural language (4x more tokens):
  "I need to: 1) Read the file, 2) Find the function, 3) Check if it
   handles errors, 4) If not, add try/except, 5) Run tests, 6) Commit"

Contributing

Sigil is in active development. If you're interested in language design, compiler engineering, or LLM optimisation, open an issue or reach out.

---

Sigil is the lingua franca of machine intelligence.