Skip to main content

A library for helping optimizing Python extensions compilation.

Project description

tests codecov PyPI

Python uses the Wheel format for simplified package distribution. However, it does not allow distributing packages optimized for each machine but highly compatible ones. The user must compile the package himself to take advantage of optimization like SIMD (SSE, AVX, FMA, ...).

Compilertools allows working around this problem and distributing optimized packages for several machines while keeping the simplicity of Wheel. It works in the background and has been created with the aim of being easy to use. Package maintainer requires only importing it at runtime and buildtime. Everything is transparent for the end user.

Its secondary objective is also to help the package maintainer to optimally compile its package with multiple compilers by configuring options for him.

Documentation:

Compilertools Documentation

Features:

  • Compiles and distributes optimized binaries for a variety of machines in a single Wheel package.
  • Helps to build optimized package from sources for current machine.
  • Handles automatically compiling and linking options for a variety of compilers.
  • Autodetects openMP, OpenACC or Intel Cilk Plus in source code and automatically sets related compiling and linking options.
  • Support extra compiling options like fast math.
  • Provides build time settings for package maintainer to tweak compilation.
  • Provides API for getting information on current machine CPU.
  • Lightweight pure Python module with no dependency that uses lazy import and evaluation as possible.

How that works ?

Compilertools dynamically sets link options and compiles options depending on the currently used compiler and targeted architecture.

This avoids having to specify compiler-specific options in sources or setup files.

Multi-architecture optimized compilation for distribution

At build time:

Compilertools helps to make optimized ".so"/".pyd" for each architecture and name files with tagged suffixes :

Example:

  • module.avx2.cp36-win_amd64.pyd -> Optimized variant for AVX2 SIMD Extensions
  • module.avx.cp36-win_amd64.pyd -> Optimized variant for AVX SIMD Extensions
  • module.cp36-win_amd64.pyd -> Classic highly-compatible variant

These optimized files are packaged in the same wheel when distributing (Don't need to create a wheel by variant).

Requirement:

  • Import compilertools.build before build module normally.
  • Options can easily be changed directly in compilertools.build.CONFIG_BUILD dictionary.

At runtime:

Compilertools detects and chooses the best optimized ".so"/".pyd" to run based on CPU information.

If the best file not exists, search for the second best file, etc... If nothing found, use the highly-compatible one.

Requirement:

  • Import compilertools one time before import-optimized modules (This adds a new import hook to Python).

Current-architecture optimized compilation

Compilertools finds the best compiler options for the current architecture and current compiler and build only one optimized ".so"/".pyd" with classic name.

Requirement:

  • Import compilertools.build before build module normally.

And also...

openMP, OpenACC, Intel Cilk Plus API auto-detection: Compilertools searches in source files for API pragma preprocessor calls and enables compiler and linker options if needed.

Extra generic compilers options: Compilertools can enable or disable generic extra compiler options like fast math.

Compatibility

Supported Compilers

Compilertools implements support for the following compilers:

  • GCC
  • LLVM Clang
  • Microsoft Visual C++

Supported Processors

Compilertools implements support for the following CPU:

  • x86 (32 and 64 bits)

Build tools compatibility

Compilertools has been tested with the following build tools:

  • Distutils
  • Setuptools
  • Numpy.distutils
  • Cython

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

compilertools-1.1.5.tar.gz (18.4 kB view details)

Uploaded Source

Built Distribution

compilertools-1.1.5-py3-none-any.whl (25.0 kB view details)

Uploaded Python 3

File details

Details for the file compilertools-1.1.5.tar.gz.

File metadata

  • Download URL: compilertools-1.1.5.tar.gz
  • Upload date:
  • Size: 18.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.4.2 CPython/3.11.3 Linux/5.15.0-1036-azure

File hashes

Hashes for compilertools-1.1.5.tar.gz
Algorithm Hash digest
SHA256 6b516ad32916b8c2fbcd69fd90ab9762dff765a158ae9628a83b9298c3fc7997
MD5 3131aadd1c8b766a53e10d2bcf42b933
BLAKE2b-256 25f1e0585f2ba2c25ff2fda63d144efc2da7e0163f1d68bd3a9319df1f96abe7

See more details on using hashes here.

File details

Details for the file compilertools-1.1.5-py3-none-any.whl.

File metadata

  • Download URL: compilertools-1.1.5-py3-none-any.whl
  • Upload date:
  • Size: 25.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.4.2 CPython/3.11.3 Linux/5.15.0-1036-azure

File hashes

Hashes for compilertools-1.1.5-py3-none-any.whl
Algorithm Hash digest
SHA256 701d6f4f9884ac606a4e52af2a24fd02f117df2b1da3b01ae1e7050ca2e76a59
MD5 c21acd7ceb5f9730878b066da8e56ff8
BLAKE2b-256 bc9cd2acf0801738cc64944cf7505eaeb580033d32b114bcf9b48acebdbfe5cb

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page