hatch-cythonize

CI/CD
Package
Meta

This is a fork of the nice hatch-cython project, aimed at providing continued maintenance and updates since the original is currently inactive.

Table of Contents

• Usage
• Configuration Options
• Notes
  • Templating (Tempita)
• License

Usage

The build hook name is cython.

• pyproject.toml

# [tool.hatch.build.hooks.cython]
# or
# (hatch.toml)
# [build.targets.wheel.hooks.cython]
# or
[tool.hatch.build.targets.wheel.hooks.cython]
dependencies = ["hatch-cythonize"]

# [tool.hatch.build.hooks.cython.options]
# or
# (hatch.toml)
# [build.targets.wheel.hooks.cython.options]
# or
[tool.hatch.build.targets.wheel.hooks.cython.options]
# include .h or .cpp directories
includes = []
# include numpy headers
include_numpy = false
include_pyarrow = false

# include_{custom}
include_somelib = {
    # must be included in build-dependencies
    pkg = "somelib",
    # somelib.gets_include() -> str
    include = "gets_include",
    # somelib.gets_libraries() -> list[str]
    libraries = "gets_libraries",
    # somelib.gets_library_dirs() -> list[str]
    library_dirs = "gets_library_dirs",
    # somelib.some_setup_op() before build
    required_call = "some_setup_op"
}

compile_args = [
  # single string
  "-v",
  # by platform
  { platforms = ["linux", "darwin"], arg = "-Wcpp" },
  # by platform & arch
  { platforms = "darwin", arch = "x86_64", arg = "-arch x86_64" },
  { platforms = ["darwin"], arch = "arm64", arg = "-arch arm64" },
  # with pep508 markers
  { platforms = ["darwin"], arch = "x86_64", arg = "-I/usr/local/opt/llvm/include", depends_path = true, marker = "python_version <= '3.10'"  },
]

directives = { boundscheck = false, nonecheck = false, language_level = 3, binding = true }

compile_kwargs = { }

• hatch.toml

# [build.hooks.cython]
# or
[build.targets.wheel.hooks.cython]
dependencies = ["hatch-cythonize"]

# [build.hooks.cython.options]
# or
[build.targets.wheel.hooks.cython.options]
directives = { boundscheck = false, nonecheck = false, language_level = 3, binding = true }
compile_args = [
    "-O3",
]
includes = []
include_numpy = false
# equivalent to include_pyarrow = true
include_somelib = { pkg = "pyarrow", include="get_include", libraries="get_libraries", library_dirs="get_library_dirs", required_call="create_library_symlinks" }
define_macros = [
    # ["ABC"] -> ["ABC", "FOO"] | ["ABC", "DEF"]
    ["NPY_NO_DEPRECATED_API", "NPY_1_7_API_VERSION"],
]

Configuration Options

Files

[build.targets.wheel.hooks.cython.options.files]
exclude = [
    # anything matching no_compile is ignored by cython
    "*/no_compile/*",
    # note - anything "*" is escaped to "([^\s]*)" (non whitespace).
    # if you need an actual * for python regex, use as below:
    # this excludes all pyd or pytempl extensions
    "([^.]\\*).(pyd$|pytempl$)",
    # only windows
    { matches = "*/windows", platforms = ["linux", "darwin", "freebsd"] },
    # only darwin
    { matches = "*/darwin", platforms = ["linux", "freebsd", "windows"] },
    # only linux
    { matches = "*/linux", platforms = ["darwin", "freebsd", "windows"] },
    # only freebsd
    { matches = "*/freebsd", platforms = ["linux", "darwin", "windows"] }
]
aliases = {"abclib._filewithoutsuffix" = "abclib.importalias"}

Explicit Build Targets

If explicit targets are required (i.e. hatch-cythonize only builds the files specified), use options.files.targets. Specifying this option will implicly enable compile_py, in addition to checking all c, cpp, and cc files against the specified inclusions.

[build.targets.wheel.hooks.cython.options.files]
targets = [
  # using a single string
  "*/compile.py",
  # or a match clause
  { matches = "*/windows", platforms = ["windows"] },
  { matches = "*/posix", platforms = ["darwin", "freebsd", "linux"] },
]

sdist

Sdist archives may be generated normally. hatch must be defined as the build-system build-backend in pyproject.toml. As such, hatch will automatically install hatch-cythonize, and perform the specified e.g. platform-specific adjustments to the compile-time arguments. This allows the full build-process to be respected, and generated following specifications of the developer.Note: If hatch-cythonize is specified to run outside of a wheel-step processes, the extension module is skipped. As such, the .c & .cpp, as well as templated files, may be generated and stored in the sdist should you wish. However, there is currently little purpose to this, as the extension will likely have differed compile arguments.

Templating

Cython tempita is supported for any files suffixed with .in, where the extension output is:

• .pyx.in
• .pyd.in
• .pyi.in For these files, expect the output .pyx.in -> .pyx. Thus, with aliasing this would look like:

[build.targets.wheel.hooks.cython.options.files]
aliases = {"abclib._somemod" = "abclib.somemod"}

• 1. Source files somemod.pyi.in, _somemod.pyx.in
• 2. Processed templates somemod.pyi, _somemod.pyx
• 3. Compiled module abclib.somemod{.pyi,.pyx}

An example of this is included in:

• pyi stub file
• pyx cython source file
• pyi stub (rendered)
• pyx cython source (rendered)

Template Arguments

Per-file matched namespaces are supported for templating. This follows the above platforms, arch, & marker formats, where if supplied & passing the condition the argument is passed to the template as a named series of keyword arguments.

If an index value is provided, and all other kwargs to templates are keywords for each index value. Follows FIFO priority for all keys except global, which is evaluated first and overriden if there are other matching index directives. The engine will attempt to merge the items of the keywords, roughly following:

args = {
    "index": [
        {"keyword": "global", ...},
        {"keyword": "thisenv", ...},
    ],
    "global": {"abc": 1, "other": 2},
    "thisenv": {"other": 3},
}

merge(args) -> {"abc": 1, "other": 3}

In hatch.toml:

[build.targets.wheel.hooks.cython.options.templates]
index = [
  {keyword = "global", matches = "*" },
  {keyword = "templated_mac", matches = "templated.*.in",  platforms = ["darwin"] },
  {keyword = "templated_mac_py38", matches = "templated.*.in",  platforms = ["darwin"], marker = "python == '3.8'" },
  {keyword = "templated_win", matches = "templated.*.in",  platforms = ["windows"] },
  {keyword = "templated_win_x86_64", matches = "templated.*.in",  platforms = ["windows"], arch = ["x86_64"] },

]

# these are passed as arguments for templating

# 'global' is a special directive reserved & overriden by all other matched values
global = { supported = ["int"] }

templated_mac = { supported = ["int", "float"] }
templated_mac_py38 = { supported = ["int", "float"] }

templated_win = { supported = ["int", "float", "complex"] }

# assuming numpy is cimported in the template
templated_win_x86_64 = { supported = ["int", "float", "np.double"]}

Notes

• MacOS users with brew installed will have brew --prefix libs and include paths added in compilation step. Code parsing is found here

• Github Runners now run MacOS on m1 platforms. You may have ci issues if you are using MacOS m1 runners and you do not disable macos-max-compat in hatch. e.g.

  # hatch.toml
  
  [build.targets.wheel]
  macos-max-compat = false
  

Development

Requirements

• a c / c++ compiler
• python 3.8-<=3.12
• git-cliff (pip install git-cliff)
• tasks

Scripts

• test: library & coverage
  • hatch run cov
• test: src structure example
  • task example
• test: simple structure example
  • task simple-structure
• commit: precommitt
  • task precommit

License

hatch-cythonize is distributed under the terms of the MIT license.