scissiparity 0.1.0

Zero-input-serialization, GIL-free parallel map-reduce.

scissiparity 🦠

Zero-input-serialization, GIL-free parallel map-reduce.

Standard Python multiprocessing is a nightmare of PicklingErrors, explicit worker pools, and massive memory duplication because it forces you to serialize data in both directions. scissiparity solves this through asymmetric serialization: it natively clones the Python interpreter's memory space via OS-level Copy-On-Write to read inputs for free, and only serializes the strictly necessary outputs.

uv add scissiparity

The Difference

Concept	multiprocessing.Pool	scissiparity
Input Propagation	Serializes/Pickles inputs across IPC queues.	Zero-serialization. Reads parent memory natively via os.fork().
Output Extraction	Pickles results via IPC queues.	Pickles results via native os.pipe() (dill).
Memory Overhead	Duplicates all inputs.	Zero-copy for inputs. OS-level Copy-On-Write mechanics.
Configuration	Requires manual chunking and worker sizing.	Zero config. Inferred purely from OS CPU topography.

Core Mechanics

• Asymmetric Serialization (Copy-On-Write): Because it leverages standard UNIX os.fork(), child processes can read the parent's massive in-memory variables instantly. You can process objects that are entirely unpicklable. Serialization (dill) is invoked only to pipe the final yielded results back to the parent.
• Exception Teleportation: If a child process raises an Exception, the error is natively serialized, piped back to the parent, and re-raised in the main thread. If one child dies, the workflow aborts gracefully. Zero "ghost compute."