Skip to main content

A JIT compiler wrapper for CPython

Project description

Pyjion

A JIT extension for CPython that compiles your Python code into native CIL and executes it using the .NET 5 CLR.

Compile and test C++ package Documentation Status

Installing

Prerequisites:

  • CPython 3.9.0
  • CMake 3.2 +
  • .NET 5
 $ git clone git@github.com:Microsoft/pyjion --recurse-submodules
 $ cd pyjion
 $ python -m pip install .

Using Pyjion

To get started, you need to have .NET 5 installed, with Python 3.9 and the Pyjion package (I also recommend using a virtual environment).

After importing pyjion, enable it by calling pyjion.enable() which sets a compilation threshold to 0 (the code only needs to be run once to be compiled by the JIT):

>>> import pyjion
>>> pyjion.enable()

Any Python code you define or import after enabling pyjion will be JIT compiled. You don't need to execute functions in any special API, its completely transparent:

>>> def half(x):
...    return x/2
>>> half(2)
1.0

Pyjion will have compiled the half function into machine code on-the-fly and stored a cached version of that compiled function inside the function object. You can see some basic stats by running pyjion.info(f), where f is the function object:

>>> pyjion.info(half)
{'failed': False, 'compiled': True, 'run_count': 1}

You can see the machine code for the compiled function by disassembling it in the Python REPL. Pyjion has essentially compiled your small Python function into a small, standalone application. Install distorm3 first to disassemble x86-64 assembly and run pyjion.dis.dis_native(f):

>>> import pyjion.dis
>>> pyjion.dis.dis_native(half)
00000000: PUSH RBP
00000001: MOV RBP, RSP
00000004: PUSH R14
00000006: PUSH RBX
00000007: MOV RBX, RSI
0000000a: MOV R14, [RDI+0x40]
0000000e: CALL 0x1b34
00000013: CMP DWORD [RAX+0x30], 0x0
00000017: JZ 0x31
00000019: CMP QWORD [RAX+0x40], 0x0
0000001e: JZ 0x31
00000020: MOV RDI, RAX
00000023: MOV RSI, RBX
00000026: XOR EDX, EDX
00000028: POP RBX
00000029: POP R14
...

The complex logic of converting a portable instruction set into low-level machine instructions is done by .NET's CLR JIT compiler.

All Python code executed after the JIT is enabled will be compiled into native machine code at runtime and cached on disk. For example, to enable the JIT on a simple app.py for a Flask web app:

import pyjion
pyjion.enable()

from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'Hello, World!'

app.run()

FAQ

How do you pronounce "Pyjion"?

Like the word "pigeon". @DinoV wanted a name that had something with "Python" -- the "Py" part -- and something with "JIT" -- the "JI" part -- and have it be pronounceable.

How do this compare to ...

PyPy?

PyPy is an implementation of Python with its own JIT. The biggest difference compared to Pyjion is that PyPy doesn't support all C extension modules without modification unless they use CFFI or work with the select subset of CPython's C API that PyPy does support. Pyjion also aims to support many JIT compilers while PyPy only supports their custom JIT compiler.

Pyston?

Pyston is an implementation of Python using LLVM as a JIT compiler. Compared to Pyjion, Pyston has partial CPython C API support but not complete support. Pyston also only supports LLVM as a JIT compiler.

Numba?

Numba is a JIT compiler for "array-oriented and math-heavy Python code". This means that Numba is focused on scientific computing while Pyjion tries to optimize all Python code. Numba also only supports LLVM.

IronPython?

IronPython is an implementation of Python that is implemented using .NET. While IronPython tries to be usable from within .NET, Pyjion does not have a compatibility story with .NET. This also means IronPython cannot use C extension modules while Pyjion can.

Psyco?

Psyco was a module that monkeypatched CPython to add a custom JIT compiler. Pyjion wants to introduce a proper C API for adding a JIT compiler to CPython instead of monkeypatching it. It should be noted the creator of Psyco went on to be one of the co-founders of PyPy.

Unladen Swallow?

Unladen Swallow was an attempt to make LLVM be a JIT compiler for CPython. Unfortunately the project lost funding before finishing their work after having to spend a large amount of time fixing issues in LLVM's JIT compiler (which has greatly improved over the subsequent years).

Nuitka and Shedskin?

Both Nuitka and Shedskin are Python-to-C++ transpilers, which means they translate Python code into equivalent C++ code. Being a JIT, Pyjion is not a transpiler.

Will this ever ship with CPython?

Goal #1 is explicitly to add a C API to CPython to support JIT compilers. There is no expectation, though, to ship a JIT compiler with CPython. This is because CPython compiles with nothing more than a C89 compiler, which allows it to run on many platforms. But adding a JIT compiler to CPython would immediately limit it to only the platforms that the JIT supports.

Does this help with using CPython w/ .NET or UWP?

No.

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pyjion-0.0.2.tar.gz (304.0 kB view details)

Uploaded Source

Built Distributions

pyjion-0.0.2-cp39-cp39-win_amd64.whl (230.2 kB view details)

Uploaded CPython 3.9Windows x86-64

pyjion-0.0.2-cp39-cp39-macosx_10_16_x86_64.whl (236.5 kB view details)

Uploaded CPython 3.9macOS 10.16+ x86-64

pyjion-0.0.2-cp39-cp39-macosx_10_15_x86_64.whl (232.8 kB view details)

Uploaded CPython 3.9macOS 10.15+ x86-64

File details

Details for the file pyjion-0.0.2.tar.gz.

File metadata

  • Download URL: pyjion-0.0.2.tar.gz
  • Upload date:
  • Size: 304.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.32.2 CPython/2.7.15

File hashes

Hashes for pyjion-0.0.2.tar.gz
Algorithm Hash digest
SHA256 66b7d5fcd212a72f2a16bfd83d7a078ca527bf1402de9a42c106e2971feafbf4
MD5 61c8d3089dfb9020ee9bc8bed7102cc7
BLAKE2b-256 b32b28a5261f4e139242d8b4ce22d365f58c64942cc8d0e04b9196d9be17668f

See more details on using hashes here.

File details

Details for the file pyjion-0.0.2-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: pyjion-0.0.2-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 230.2 kB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.6.1 requests/2.25.0 setuptools/50.3.1 requests-toolbelt/0.9.1 tqdm/4.52.0 CPython/3.9.0rc2

File hashes

Hashes for pyjion-0.0.2-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 a5816b293830ca411cbb311a933d2a89962d6c3f9c645d40be872f74112cc093
MD5 3cac2f4ab718f9273b72be8d49c010a9
BLAKE2b-256 541c009c77aab96e9eba0ac2bc3fd6b5c0fa571ea8c2c4fb29e4d709e43ea287

See more details on using hashes here.

File details

Details for the file pyjion-0.0.2-cp39-cp39-macosx_10_16_x86_64.whl.

File metadata

  • Download URL: pyjion-0.0.2-cp39-cp39-macosx_10_16_x86_64.whl
  • Upload date:
  • Size: 236.5 kB
  • Tags: CPython 3.9, macOS 10.16+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.6.1 requests/2.25.0 setuptools/50.3.1 requests-toolbelt/0.9.1 tqdm/4.52.0 CPython/3.9.0rc2

File hashes

Hashes for pyjion-0.0.2-cp39-cp39-macosx_10_16_x86_64.whl
Algorithm Hash digest
SHA256 db3b127a69166d0734ce7b9ce65ad2ba71c2ae0cf09c344bad446f05c519cc29
MD5 e19a6d07672594b9041d04363f30a9bc
BLAKE2b-256 4071981679704cf93276d3e2398ca73db081eefa62fe4c29032a4d7b7cc842da

See more details on using hashes here.

File details

Details for the file pyjion-0.0.2-cp39-cp39-macosx_10_15_x86_64.whl.

File metadata

  • Download URL: pyjion-0.0.2-cp39-cp39-macosx_10_15_x86_64.whl
  • Upload date:
  • Size: 232.8 kB
  • Tags: CPython 3.9, macOS 10.15+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.32.2 CPython/2.7.15

File hashes

Hashes for pyjion-0.0.2-cp39-cp39-macosx_10_15_x86_64.whl
Algorithm Hash digest
SHA256 bbcac768fba0f1494e25c390990ef9ff53614266c53a67f6d25dd76d5adbb5ad
MD5 3e5730b24d1ed981389d857417f6a1c2
BLAKE2b-256 51ffa278f37460cc55cf0f43a98c12800583bd9bb432d32ae1fe8e2542fbd09c

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page