concurrent-c-node

Call Node (and npm packages) from Python. Native types, exceptions, callbacks, and async all cross the boundary.

Why use this

Call real Node from Python — require, native addons, npm — not a JS engine in-process. One child, shm for bulk, from cc_node import require in Jupyter/Colab with no %load_ext. pythonia is the packaged peer; this is faster on the wire and honest about bulk and callbacks.

Part of Concurrent-C — a strict C11-superset preprocessor: .ccs lowers to plain C and compiles with your host C compiler. (This bridge itself is pure Python stdlib — no native build.)

Map of the three boundaries (CC hosts JS, native modules, this package bridge): JS / Python interop.

from cc_node import require          # one session child (lazy)
_ = require('lodash')                # cwd node_modules
_.chunk([1, 2, 3, 4, 5], 2)          # [[1, 2], [3, 4], [5]]

A private child (create()) is still there when you want N Nodes or an explicit lifetime — not required for the first call.

import cc_node
js = cc_node.create()                # a private Node, not the session
semver = js.require('semver')
semver.satisfies('1.2.3', '^1.0.0')  # True
js.destroy()                         # or: with cc_node.create() as js:

The other direction (Python from Node): concurrent-c-python (in-process by default; vs pymport / ncp / pythonia in that README).

Every create() is a separate Node — real addons, crash isolation, measurable wire. N domains = N processes. require() / get() share one session for the process (the Jupyter kernel).

Cheat sheet

• Always a child node. A call blocks until JS answers; thenables wait in the child. No { async: true }.
• from cc_node import require is the session. create() is a private child. reset() / %js_reset / %reset / atexit tear the session down.
• Scalars / None materialize; empty {} stays a handle; everything else is a JsHandle until str() / attrs / a call.
• Notebook: import cc_node registers %%js (no %load_ext). Same session as require().
• eval() is one RTT, no extra globals. %%js / eval_cell install cwd require once.
• --bind is Object.assign(globalThis, …) of names you name (wire types only; no pickle). Missing names refuse.
• First Ctrl-C finishes the in-flight call (wire stays in sync). Interrupt again to kill the child.

pip install concurrent-c-node                 # needs node on PATH
pip install 'concurrent-c-node[jupyter]'      # IPython (%%js); require() does not need this
python -m cc_node.examples.use_node
python -m cc_node.examples.bench_wire
python -m cc_node.benchmarks.multi_domain
python -m cc_node.benchmarks.vs_alts          # vs pythonia / DIY node / pythonmonkey / mini-racer

Jupyter / Colab

Same verb as pythonia: require. No %load_ext, no create(), no destroy() for the happy path. One session child for the kernel; console.log lands in the cell.

%pip install concurrent-c-node
# if `node` is missing (typical Colab):
!apt-get install -y nodejs

from cc_node import require
require('lodash').chunk([1, 2, 3, 4, 5], 2)

import cc_node also registers %%js (IPython already running; the [jupyter] extra is only if you want magics without IPython already installed).

%%js
console.log('hi')              # shows in the cell
require('lodash').chunk([1, 2, 3, 4, 5], 2)

xs = [1, 2, 3, 4]

%%js -b xs -t chunks
xs.map(x => x * 2)             # wire types only; no pickle fallback

eval() is unchanged (one RTT, no extra globals). %%js / eval_cell install cwd require once so cells look like Node (require('path')). --bind is Object.assign(globalThis, …) — missing names and non-wire types (DataFrame, a set) fail articulately. Reserved names (require, process, globalThis, …) refuse so a bind cannot shadow Node.

Interrupt. First Ctrl-C does not abandon the in-flight reply (that would desync the wire and drop callbacks). The call finishes, the result is discarded, the domain stays up. Interrupt again to kill the child — cooperative destroy() cannot stop a JS for (;;) {}. Same honesty as the rest of the bridge.

JS kernels (tslab, Deno Jupyter) are concurrent-c-python; Colab is not that. There, default create() blocks the kernel thread — py.task / { isolated: true } when the loop must stay live.

Measured

bench_wire · cc_node_bridge_py_20260810.txt:

Wire: line-JSON on dedicated fds (stdio stays yours). Bulk spill: private 0700 dir, 0600 files, removed with the bridge.

Vs pythonia / pythonmonkey / mini-racer / DIY node

Most “JS from Python” libraries are not Node. pythonia (PyPI javascript, JSPyBridge) is the packaged peer that is: require() on import, one child. Bulk is a sum over 1M floats (.length on an in-process wrapper is free and lies). Snapshot: cc_node_vs_alts_20260813.txt · harness: benchmarks/vs_alts.py.

Tiny scalars: pythonmonkey’s in-process SM beats a child. Real Node (require('fs'), native addons, callbacks, stdout stays yours): this package — ~2× pythonia on identity, shm for bulk, Python callables are sync (pythonia’s JS side sees a Promise). Isolated node -e per call is ~1000× a persistent child. Optional engines SKIP if not importable — not package deps.

The other direction (Python from Node): concurrent-c-python vs pymport / ncp / pythonia.

Surface

• Plain data (numbers, str, bool, None, lists, non-empty dicts) by value; else a domain-owned handle (attrs, calls, str() → String()). Non-finite floats are tagged.
• Handles are per-domain. stats() / release() / idempotent destroy(); afterwards: bridge is closed.
• eval_cell(src, bindings=) is the notebook door (%%js): cwd require once, optional globalThis binds; eval() stays one RTT. get() / require() / eval() at module level share that session.
• Crash isolation, not a sandbox. destroy() is cooperative; an in-flight CPU-bound call finishes or you kill the child (bridge_stress.md).

Promises

Awaited in the child before the reply — no async/await on the Python side. Same honesty as concurrent-c-python: a call blocks until the other runtime answers.

fetchish = js.eval('async (x) => { return { doubled: x * 2 } }')
fetchish(21)   # {'doubled': 42}

Callbacks

mapped = js.eval('(f) => [1, 2, 3].map(f)')(lambda x, *rest: x * 10)
# map passes (value, index, array) — take *rest

Exceptions cross both ways with messages intact.

Buffers

bytes / array.array / 1-D numpy → typed arrays (and back). Small inline; large via shm.

import array
total = js.eval('(a) => a.reduce((s, x) => s + x, 0)')
total(array.array('d', range(1_000_000)))

Common issues

Cannot find module. require / import resolve from the Python process cwd (node_modules next to your notebook or program), not from this wheel’s site-packages, and this package does not npm install on a miss (pythonia does). npm install lodash in that directory is the fix; or create(node=…) / CC_NODE_BIN when the wrong Node is on PATH.

Empty {} stays a handle

An empty object has to stay on the Node side — a materialized Python dict would lose later property use that matches Node. So js.eval('({})') returns a live handle:

o = js.eval('({})')                    # JsHandle, not {}
js.eval('(o) => { o.x = 1; return o.x }')(o)   # 1
js.eval('({a: 1})')                    # {'a': 1} — data return

Non-empty plain objects still cross as Python dicts. Same-domain handles chain (h.update(…).digest(…)).

Wire cost vs tiny work. Round trip is ~100µs; a one-line JS helper on three numbers loses to pure Python. Prefer Python (or a native CC module) for small/hot work; use the bridge when Node/npm owns the kernel. Multi-core: python -m cc_node.benchmarks.multi_domain (~2.8× on 3 domains here).

Choosing node

1. create(node='/path/to/node')
2. CC_NODE_BIN
3. node on PATH

Packages: cwd node_modules, same as Node itself.

From Concurrent-C (not Python): cc_js_new(false, a) hosted (libnode), or cc_js_new(true, a) for this wire — jsdemo.shcc.

Publishing

PyPI Trusted Publishing (OIDC) — no API token:

1. Publishing settings: owner sreekotay, repo concurrent-c, workflow publish-cc-node.yml, environment pypi
2. GitHub Environment pypi
3. Bump pyproject.toml, push, then:

gh workflow run publish-cc-node.yml

Both bridges (npm OIDC + PyPI OIDC):

./scripts/publish_bridges.sh --publish --minor
# fallbacks: --npm-local / --pypi-twine

Examples: use_node, bench_wire, benchmarks.multi_domain, benchmarks.vs_alts. Jupyter: from cc_node import require (or import cc_node then %%js). Stress: stress/bridge/.
Own hot path in C/CC → native module (40–90ns) — JS / Python interop.