Skip to main content

Jupyter notebook verification, compilation, and DAG-based conversion for the SciTeX ecosystem

Project description

SciTeX Notebook (scitex-notebook)

PyPI Python Tests Install Test Coverage Docs License: AGPL v3

Jupyter notebook verification, compilation, and DAG-based conversion

PyPI version Documentation Tests License: AGPL-3.0


Interfaces: Python ⭐⭐⭐ · CLI ⭐⭐ · MCP ⭐⭐ · Skills ⭐⭐ · Hook — · HTTP —

Problem and Solution

# Problem Solution
1 Cell order lies — on-disk .ipynb cell sequence has no relationship to execution order, so naive jupyter nbconvert produces scripts that don't run DAG from timestamps — reconstructs the true execution dependency graph from scitex-clew session timestamps, then emits a topologically-ordered .py or a Mermaid diagram
2 Silent untracked I/Oscitex.io.save/load calls outside @stx.session leave no reproducibility trail, but nothing warns you check_notebook() — scans for untracked I/O and flags cells that bypass session tracking
3 Exploration vs. production gap — notebooks let you iterate freely, but shipping means rewriting by hand into a clean script "Do what you want, organize later" — execute cells in any order while exploring; compile_notebook(...).to_script() emits the production-ready DAG-ordered script

Problem

Jupyter notebooks can be executed in any cell order. The on-disk cell sequence often has no relationship to the actual execution dependency graph. Traditional notebook-to-script converters assume linear order and produce broken scripts.

Solution

scitex-notebook records actual execution order via scitex-clew timestamps, then reconstructs the dependency DAG afterward. "Do what you want, organize later."

  • Verify clew sessions for a notebook (reproducibility check)
  • Check for scitex.io calls missing @scitex.session decoration
  • Compile execution history into a Mermaid DAG or topologically-sorted .py
  • Convert .ipynb to a SciTeX Python script (per-cell or unified main())

Installation

Requires Python >= 3.10.

pip install scitex-notebook

Optional extras:

pip install "scitex-notebook[mcp]"     # MCP server for AI agents
pip install "scitex-notebook[linter]"  # IO-call conversion via scitex-linter
pip install "scitex-notebook[all]"     # everything

Quickstart

Python API
import scitex_notebook as notebook

cells    = notebook.parse_notebook("experiment.ipynb")
issues   = notebook.check_notebook("experiment.ipynb")     # untracked IO
results  = notebook.verify_notebook("experiment.ipynb")    # via clew DB
compiled = notebook.compile_notebook("experiment.ipynb")

print(compiled.to_mermaid())   # Mermaid DAG diagram
print(compiled.to_script())    # DAG-ordered Python script

notebook.convert_notebook(
    "experiment.ipynb",
    output="experiment.py",
    mode="unified",            # or "per_cell"
)
CLI
scitex-notebook verify experiment.ipynb
scitex-notebook check experiment.ipynb
scitex-notebook compile experiment.ipynb --format mermaid
scitex-notebook compile experiment.ipynb --format script -o experiment.py
scitex-notebook convert experiment.ipynb --mode unified -o experiment.py
MCP Server — for AI Agents
Tool Description
notebook_verify Verify clew sessions for a notebook
notebook_check Flag untracked scitex.io calls
notebook_compile Return Mermaid DAG / script / JSON
notebook_convert Convert .ipynb to .py
python -m scitex_notebook.mcp_server

Dependencies

  • Required: scitex-clew — execution-order reconstruction via timestamped sessions.
  • Optional: scitex-linter — advanced IO-call rewriting during conversion.

Part of SciTeX

SciTeX Notebook is part of SciTeX.

The SciTeX system follows the Four Freedoms for Research below, inspired by the Free Software Definition:

Four Freedoms for Research

  1. The freedom to run your research anywhere — your machine, your terms.
  2. The freedom to study how every step works — from raw data to final manuscript.
  3. The freedom to redistribute your workflows, not just your papers.
  4. The freedom to modify any module and share improvements with the community.

AGPL-3.0 — because we believe research infrastructure deserves the same freedoms as the software it runs on.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scitex_notebook-0.1.1.tar.gz (32.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

scitex_notebook-0.1.1-py3-none-any.whl (24.9 kB view details)

Uploaded Python 3

File details

Details for the file scitex_notebook-0.1.1.tar.gz.

File metadata

  • Download URL: scitex_notebook-0.1.1.tar.gz
  • Upload date:
  • Size: 32.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for scitex_notebook-0.1.1.tar.gz
Algorithm Hash digest
SHA256 cdedd0e1e50e4f40e7aa281fe54687ee2c94df8cfb53459431b0b04a2372b8e1
MD5 e4bd551e8aec25f5ef58ae4cc2ed6066
BLAKE2b-256 2294c651811a50672f29c491a92f019ae44cb6819d41fdc567ed666acbbd04a6

See more details on using hashes here.

Provenance

The following attestation bundles were made for scitex_notebook-0.1.1.tar.gz:

Publisher: publish-pypi.yml on ywatanabe1989/scitex-notebook

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file scitex_notebook-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: scitex_notebook-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 24.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for scitex_notebook-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 5abcc6bcc06407e88a7b7ed1985b1ad28e0eeb272dafc4681c0e59272fe8f540
MD5 92a8db0f4b6a99d5cb382c28411f41f6
BLAKE2b-256 5cf508483abe22281403ade4bb670184771dde87e42c04fef89b80eeb1b2a3f9

See more details on using hashes here.

Provenance

The following attestation bundles were made for scitex_notebook-0.1.1-py3-none-any.whl:

Publisher: publish-pypi.yml on ywatanabe1989/scitex-notebook

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page