cython-plus 0.3.0

Multi-core concurrent programming in Python, based on the Cython language

Warning

This is the README for the Cython+. Since Cython+ is heavily based on Cython, you may want to read also the README for Cython.*

What’s Cython+?

The aim of the “Cython+” project is to ensure that all the cores of a microprocessor can be efficiently exploited with a program written in Python in the field of system or network programming, so as to correct the main shortcoming of the Python language and increase the competitiveness of its ecosystem. The envisaged approach consists in transferring the extremely powerful multi-core concurrent programming model of the Go language to the Python language by relying on innovative scientific and technological approaches stemming from three decades of French know-how in the field of concurrent object programming around Actalk, and leveraging the existing Cython language.

Cython is actually a superset of the Python language that brings together the strong typing of Python, performance equivalent to C and a form of low-level parallelism well suited to scientific computing. It is with Cython that the scikit-learn libraries or certain components of the NEO transactional distributed database are developed. Cython corrects the shortcomings of the Python language in terms of typing or performance. Cython also corrects the Global Interpreter Lock (GIL) problem which is at the origin of the poor support of multi-core microprocessors in Python.

In the “Cython+” project, we propose to remove the GIL in a very specific way: only at the level of asynchronous Cython functions not calling Python objects. So nothing is changed in the Python language nor in the “CPython” runtime which is the reference implementation of the Python language in C. All programs already developed in Python remain compatible. We only modify the Cython compiler and the subpart of the Cython language disjoint from Python, which we extend with a garbage collector and coroutines that can be used on a multi-core architecture.

Thus, “Cython+” will offer the same kind of coroutine programming as Go, the same level of parallelism, the same kind of memory management, the same kind of performance, exception handling that Go does not fully benefit from, a better concurrent programming model than Go, a very well-stocked standard library with much broader community support, and guaranteed memory isolation between threads. “Cython+” will become an alternative to Go with many advantages, strengthening the community of the leading development language that Python has become.

Installation and basic use

Simply run:

pip install cython-plus

Then you can use the habitual Cython commands: cython, cythonize and cygdb, as well as import the cython module from your Python code.

Only Python 3.9 on Linux is supported at the moment. It doesn’t (currently) work on MacOS or Windows.

Documentation

• Project Website: <https://www.cython.plus/>

• Documentation:

  • Motivation

  • Basic Syntax (by example)

  • Interacting with Python (by example)

  • Concurrency

• Blog posts and articles:

  • Automatic multithreaded-safe memory managed classes in Cython

  • HowTo Use Cython+ in Jupyter Notebook

• Sandbox (various code snippets and benchmark to help you get started): <https://github.com/abilian/cythonplus-sandbox>

Development

• Project repository: <https://lab.nexedi.com/nexedi/cython>

• Alternate repository (read-only): <https://github.com/abilian/cythonplus>

• CI: <https://github.com/abilian/cythonplus/actions>

License & Copyright

Cython+ is a (friendly) fork of Cython.

Its copyright belongs to the Cython original authors (as listed Here) as well as to the members of the Cython+ consortium: Nexedi, Abilian, Teralab and Inria.

Cython+ is licensed under the permissive Apache License. See LICENSE.txt.

Contributing

Want to contribute to the Cython+ project?

Please contact us at <https://www.cython.plus/contact/>.

Acknowledgements

Cython+ is based on the work of the Cython authors.

The Cython+ project has been selected to receive funding from the PSPC-Régions 1. It is supported by Cap Digital and financed by the PIA and the Paris Region.