cist-kernel 0.0.64

CLI for cist kernel telemetry and compilation

CIST Monorepo

Polyglot monorepo for the Contract Instruction Set (.cis) language.

This repository ships three user-facing pieces together:

• cist-kernel Python package — the CLI, the compiler, and the cloud compilation client.
• A VS Code extension bundled inside the package — cist init installs it into your local VS Code extensions folder. Recognises both .cis source files and .mex compiled binaries.
• A build orchestrator (scripts/build.py) for cutting releases.

Language Snapshot

use math
use compute
use risk

market_data := from data (source: "binance", symbols: ["BTC-USD"], resolution: "1m")

sma_20 := math.sma(market_data.close, 20)
signal := compute.crossover(market_data.price, sma_20)

if signal and position.size == 0:
	execute(market_buy(size: risk.calculate_size(portfolio_value: 10000)))
elif position.pnl < -100:
	execute(close_position())

Repository Layout

cist/
├── pyproject.toml
├── cist.toml                   ← workspace config (created by cist init)
├── README.md
├── .gitignore                  ← *.mex and .env excluded by default
├── scripts/
│   └── build.py
├── cist_cli/
│   ├── __init__.py
│   ├── main.py
│   ├── network.py
│   ├── compiler/               ← NEW: compilation pipeline
│   │   ├── __init__.py
│   │   ├── lexer.py            ← CISTCompiler: syntax + semantic validation → IR
│   │   └── mex.py              ← MexWriter / MexReader: .mex binary format
│   ├── workspace/              ← NEW: workspace scaffold
│   │   ├── __init__.py
│   │   ├── init.py             ← init_workspace(): creates directory structure
│   │   └── templates.py        ← all generated-file templates
│   └── assets/
│       └── *.vsix              ← bundled VS Code extension
├── tests/
│   ├── test_compiler.py
│   ├── test_mex.py
│   ├── test_workspace.py
│   ├── test_entrypoint.py
│   ├── test_build_script.py
│   └── test_vscode_grammar.py
└── extensions/
    ├── package.json            ← declares .cis and .mex language contributions
    ├── language-configuration.json
    └── syntaxes/
        └── cis.tmLanguage.json

Each contract lives in its own sub-directory under contracts/:

contracts/
└── {contract_name}/
    ├── {contract_name}.cis     ← main source file
    ├── program.mex             ← compiled binary (created by cist compile)
    └── bench/
        └── example.cis         ← test/bench scenarios (extend freely)

Getting Started

1. Install or upgrade

python -m pip install --upgrade cist-kernel

The --upgrade flag matters. A plain pip install cist-kernel keeps an existing older version and will not pull in new features.

If cist is not on your PATH after install, use the fallback:

python -m cist_cli

On Windows, the installer will print the exact PowerShell one-liner to add the scripts folder to your PATH permanently the first time you run via python -m cist_cli.

2. Initialise a workspace (if you haven't already)

cist init              # in the current directory
cist init --path ~/my-workspace

cist init creates the following structure from scratch (safe to re-run — existing files are never overwritten):

<workspace>/
├── cist.toml                    project name, kernel version, compiler settings
├── .env                         CIST_CLOUD_API_KEY placeholder
├── .gitignore                   *.mex and .env pre-seeded
└── contracts/
    └── example/
        ├── example.cis          starter contract (edit freely)
        └── bench/
            └── example.cis      bench / test scenarios (extend freely)

It also silently extracts the bundled VS Code extension into every detected editor extensions folder (.vscode, .vscode-insiders, .antigravity), removing any stale legacy versions. Reload the window once with Ctrl+Shift+P → Developer: Reload Window to activate syntax highlighting for .cis and .mex files.

3. Add your API key

Open the .env file and fill in your key:

CIST_CLOUD_API_KEY=sk_live_your_key_here

4. Write your contract

Edit contracts/example/example.cis or create a new sub-directory:

use math
use hyper

contract OilPriceMonitor:
    which defines:
        while True:
            oil_price = hyper.get_price("OIL")

            then:
                if oil_price > 80.0:
                    print("Consider selling.")

Add test scenarios in contracts/example/bench/:

contract BenchHighPrice:
    which defines:
        oil_price = 90.0
        then:
            if oil_price > 80.0:
                print("bench: high price path OK")

5. Compile

cist compile contracts/example/example.cis

What happens:

1. Syntax check — the compiler validates brackets, strings, use imports, from data(…) expressions, indentation, and scans contract and write fn declarations.
2. Binary emit — if the file is valid, program.mex is written next to the source file. This binary contains the compressed IR; the raw source is never stored inside it, and the file is excluded from version control via .gitignore.
3. Cloud stream — the IR is streamed to the CIST Cloud compiler. Logs, artifact IDs, and errors are printed as they arrive.

✓ example.cis  42 lines · modules: math, hyper · 0 data stream(s)
✓ Binary written → contracts/example/program.mex
Connecting to CIST Cloud...

Add --deploy to mark the build for live execution:

cist compile contracts/example/example.cis --deploy

Add --verbose to see the full module and data-stream breakdown:

cist compile contracts/example/example.cis --verbose

---

CLI Reference

cist init [--path <dir>]

Initialises a CIST workspace.

• Creates contracts/{name}/{name}.cis and contracts/{name}/bench/example.cis.
• Writes cist.toml (project name, kernel version, compiler settings).
• Writes .env stub with CIST_CLOUD_API_KEY placeholder.
• Seeds .gitignore with *.mex and .env (appends if the file already exists; never duplicates).
• Removes stale legacy extension versions and silently extracts the bundled .cis + .mex VS Code extension.

cist compile <file.cis> [--deploy] [--verbose]

Compiles a .cis contract.

• Step 1 — local validation: bracket matching, quote closure, use imports, from data(…) expressions, mixed tab/space indentation, UTF-8 encoding.
• Step 2 — binary emit: writes a program.mex binary alongside the source. The binary is zlib-compressed, integrity-checked with CRC-32, and does not contain the original source text.
• Step 3 — cloud stream: streams the compiled IR to the CIST Cloud. Logs, artifact IDs, and errors are printed in real time.
• --deploy flags the build as live.
• --verbose prints the full module list and data-stream count.

cist sync-vscode-extension

Re-installs or updates the bundled VS Code extension without running a full init.

---

The .mex Binary Format

When cist compile succeeds it writes a program.mex file alongside the source. The format is:

Offset	Size	Content
0	4 B	Magic: MEX\x00
4	2 B	Format version (big-endian uint16, currently 1)
6	4 B	CRC-32 of the compressed payload (big-endian uint32)
10	N B	zlib-compressed UTF-8 JSON payload

The payload holds the compiler's IR (module list, contract names, function names, data stream count, line count, warnings). The original source text is stripped before writing.

.mex files are recognised by the VS Code extension (file-tree icon, language ID mex) but are intentionally non-editable binary blobs. They are excluded from version control via the workspace .gitignore.

---

Configuration

cist.toml

Created by cist init at the workspace root. Controls project metadata and compiler behaviour:

[project]
name = "example"
version = "0.1.0"
description = ""

[kernel]
version = "0.0.38"   # kernel version this workspace was initialised with

[compiler]
strict = false        # set true to treat warnings as errors

Environment Variables

Loaded from .env (if present) and from the shell environment.

Variable	Required	Default	Purpose
CIST_CLOUD_API_KEY	Yes (for compile)	—	Bearer token for the cloud compiler
CIST_CLOUD_ENDPOINT	No	https://api.cist.io/v1	Override the cloud endpoint
CIST_SKIP_VSCODE_INSTALL	No	—	Set to 1 to skip extension install (useful in CI)
VSCODE_EXTENSIONS	No	—	Override the VS Code extensions root directory

---

Build Artifacts

Build both deliverables with:

python scripts/build.py

Expected outputs:

• dist/cist_kernel-*.whl — Python wheel containing the CLI and the bundled .vsix.

The build script verifies that the VSIX version inside the wheel matches the project version in pyproject.toml before declaring success.

---

Notes

• Local validation is lightweight and non-executing. It never runs contract code.
• Cloud compilation requires CIST_CLOUD_API_KEY. The .env file is loaded automatically.
• The VS Code grammar uses Python-style scope categories with CIST-specific handling for use, contract, write fn, which defines:, then:, and from data(…).
• The cist_cli/compiler/ and cist_cli/workspace/ packages are independently importable for use in tooling or test fixtures.