SciTeX Notebook (scitex-notebook)

Jupyter notebook verification, compilation, and DAG-based conversion to topologically-ordered Python scripts.

Full Documentation · uv pip install scitex-notebook[all]

Problem and Solution

Demo

%%{init: {'flowchart': {'nodeSpacing': 20, 'rankSpacing': 40, 'curve': 'linear'}, 'themeVariables': {'fontSize': '12px'}}}%%
flowchart LR
    A["experiment.ipynb"] --> B[parse_notebook]
    B --> C[scitex-clew DB timestamps]
    C --> D[compile_notebook DAG]
    D --> E["to_mermaid()"]
    D --> F["to_script() topologically ordered .py"]

_{Figure 1. Notebook compilation from parse to DAG-ordered script.}

Out-of-order cells in the notebook are re-ordered into a runnable script:

$ scitex-notebook compile-notebook experiment.ipynb --format script -o experiment.py
$ python experiment.py     # runs cleanly, every time

Installation

uv pip install "scitex-notebook[all]"

Requires Python >= 3.10.

uv pip install "scitex-notebook[mcp]"     # MCP server for AI agents
uv pip install "scitex-notebook[linter]"  # IO-call conversion
uv pip install -e ".[dev]"                 # editable install

Architecture

1. Parse and verify

parse reads cells while verify and check scan clew sessions and untracked I/O.

2. Compile and convert

compile builds the timestamp DAG and convert emits topologically-ordered scripts.

3. Serve and extend

mcp_server exposes notebook tools and the IPython magic tracks live cells.

%%{init: {'flowchart': {'nodeSpacing': 20, 'rankSpacing': 40, 'curve': 'linear'}, 'themeVariables': {'fontSize': '12px'}}}%%
flowchart LR
    PARSE[parse module] --> VERIFY[verify and check]
    VERIFY --> COMPILE[compile DAG]
    COMPILE --> CONVERT[convert script]
    COMPILE --> MCP[mcp server]
    CONVERT --> OUT[runnable outputs]
    MCP --> OUT

_{Figure 2. Module collaboration from parse to served outputs.}

Four Interfaces

import scitex_notebook

cells    = scitex_notebook.parse_notebook("experiment.ipynb")
issues   = scitex_notebook.check("experiment.ipynb")         # untracked IO
results  = scitex_notebook.verify("experiment.ipynb")        # via clew DB
compiled = scitex_notebook.compile("experiment.ipynb")

print(compiled.to_mermaid())   # Mermaid DAG diagram
print(compiled.to_script())    # DAG-ordered Python script

scitex_notebook.convert(
    "experiment.ipynb",
    output="experiment.py",
    mode="unified",            # or "per_cell"
)

scitex-notebook verify-notebook experiment.ipynb
scitex-notebook check-notebook experiment.ipynb
scitex-notebook compile-notebook experiment.ipynb --format mermaid
scitex-notebook compile-notebook experiment.ipynb --format script -o experiment.py
scitex-notebook convert-notebook experiment.ipynb --mode unified -o experiment.py
scitex-notebook --help-recursive               # all commands + flags
scitex-notebook --json verify-notebook exp.ipynb  # structured output

scitex-notebook mcp start                      # start stdio server
scitex-notebook mcp list-tools                 # list available tools
scitex-notebook mcp doctor                     # verify MCP dependencies

Python API

from scitex_notebook import (
    parse_notebook, get_code_cells, get_notebook_name,
    compile, convert, verify, check,
)

cells = parse_notebook("experiment.ipynb")
compiled = compile("experiment.ipynb")
print(compiled.to_mermaid())   # Mermaid DAG
print(compiled.to_script())    # Topologically-ordered .py

results = verify("experiment.ipynb")
issues = check("experiment.ipynb")

CLI

scitex-notebook compile-notebook experiment.ipynb   # DAG-ordered .py (--format script)
scitex-notebook convert-notebook experiment.ipynb   # .ipynb → @stx.session script
scitex-notebook verify-notebook experiment.ipynb    # clew session pass/fail
scitex-notebook check-notebook experiment.ipynb     # untracked-IO scan
scitex-notebook list-python-apis -vv                # public APIs with signatures
scitex-notebook skills list                         # bundled skill pages
scitex-notebook --help-recursive                    # full CLI reference

IPython Magic Extension

%load_ext scitex_notebook   # one line at the top of any notebook

Once loaded, every executed cell is analysed at runtime:

• Hidden-state leak detection — warns when a cell reads a name not defined by any earlier cell in this run
• Out-of-order execution check — flags non-monotonic execution_count
• Untracked I/O — every stx.io.save/load call is recorded per-cell

Cell metadata (dependencies, warnings, file hashes) is written to the same Clew store used by @scitex.session and stx.io.

%load_ext scitex_notebook
%unload_ext scitex_notebook

Dependencies

• Required: scitex-clew — execution-order reconstruction via timestamped sessions.
• Optional: scitex-linter — advanced IO-call rewriting during conversion.

Part of SciTeX

scitex-notebook is part of SciTeX. Install via the umbrella with pip install scitex[notebook] to use as scitex.notebook (Python) or scitex notebook ... (CLI).

The SciTeX system follows the Four Freedoms for Research below, inspired by the Free Software Definition:

Four Freedoms for Research

0. The freedom to run your research anywhere — your machine, your terms.
1. The freedom to study how every step works — from raw data to final manuscript.
2. The freedom to redistribute your workflows, not just your papers.
3. The freedom to modify any module and share improvements with the community.

AGPL-3.0 — because we believe research infrastructure deserves the same freedoms as the software it runs on.