citry-core 1.3.0

Singular Python API for Rust code used by Citry

citry_core

Python package that exposes Rust functionality from the twin citry_core_py Rust crate.

Overview

This package is the Python-side twin to the citry_core_py Rust crate. The connection between them is defined in pyproject.toml via the [tool.maturin] section:

[tool.maturin]
manifest-path = "../../../crates/citry_core_py/Cargo.toml"
module-name = "citry_core._rust"

When built with maturin, the Rust crate is compiled into a Python extension module that provides the core functionality, while this package adds Python-side wrappers, type stubs, and additional utilities.

Maturin module-name configuration

Important: The module-name setting is crucial for this package to work correctly.

By default, maturin would create a Python module with the same name as the Rust crate (citry_core_py), but we want to publish this package as citry_core. The module-name = "citry_core._rust" setting tells maturin to:

1. Create the compiled Rust extension as _rust.cpython.so (instead of citry_core_py.cpython.so)
2. Place it in the citry_core/ directory (matching our Python package name)
3. Merge it with our existing citry_core/ Python code

This allows us to:

• Keep the Rust crate named citry_core_py (useful for potential future bindings like citry_core_js)
• Publish the Python package as citry_core (the desired public name)
• Have maturin automatically merge the Rust binary with our Python code

The module-name format is <python_package_name>.<rust_module_name>, where:

• python_package_name must match the directory name in src/ (e.g., src/citry_core/)
• rust_module_name must match the #[pymodule] function name in the Rust crate's lib.rs (e.g., fn _rust)

See the detailed comments in pyproject.toml for more information about this configuration.

Package structure

The package uses a src layout:

packages/py/citry_core/
├── citry_core/               # Package code
│   ├── __init__.py           # Empty - see below
│   ├── _rust.py              # Python module binary generated by maturin
│   │                         # from `crates/citry_core_py`.
│   │                         # Name set by `module-name` setting in `pyproject.toml`
│   ├── _rust.pyi             # Type stubs for Rust API
│   │                         # Filename matches <rust_module_name> from `module-name`
│   │
│   │   # API for individual Rust crates defined
│   │   # as separate submodules / subdirectories
│   │
│   ├── html_transform/      # HTML transformation API
│   │   └── __init__.py      # API for this submodule
│   ├── safe_eval/           # Safe eval API
│   ├── template_parser/     # Template parsing API
│   └── ...
├── tests/                    # Package tests
└── pyproject.toml

Manual API management

By default, maturin auto-generates __init__.py files. However, this package manually manages the Python-side API for several reasons:

1. Multiple crates: The Rust crate exposes multiple submodules (html_transform, safe_eval, template_parser). We split these into separate submodules to avoid name conflicts.
2. Python-side post-processing: Some crates need Python-side code that wraps or extends the Rust API
   • For example, both safe_eval and template_parser add a layer to call exec() on the generated code.

The root __init__.py is intentionally empty because we namespace each Rust crate under its own Python module. Instead, the API for each crate is defined in a separate directory with its own __init__.py file, e.g.:

• citry_core/html_transform/__init__.py
• citry_core/safe_eval/__init__.py
• citry_core/template_parser/__init__.py

This allows clean, namespaced imports:

from citry_core.html_transform import transform_html
from citry_core.safe_eval import safe_eval
from citry_core.template_parser import parse_tag

Remember: When you import
citry_core.html_transform,
you are directly accessing
src/citry_core/html_transform/__init__.py
You are NOT accessing
src/citry_core/__init__.py

Type stubs and runtime access

Issues with type checking

Type checkers (mypy, Pylance/Pyright) cannot pick up signatures and docstrings from the Rust API generated by maturin. The Rust code is compiled into a binary extension module, and type checkers don't have access to the original Rust source code or its PyO3 annotations.

To solve the type checking issue, we define type stubs in _rust.pyi.

The _rust.pyi:

• Re-defines function signatures and docstrings
• Enables type checking, IDE autocomplete, hover tooltips, etc
• Important: The filename _rust.pyi MUST match the second part of the module-name setting in pyproject.toml
  (tool.maturin.module-name = "citry_core._rust").
  This ensures type checkers can find the type stubs for the Rust extension module.

Issues with Pytest and virtual modules

Another issue is with pytest. We define nested Python submodules for each Rust crate to avoid name conflicts in the Rust API. But pytest can't import the nested modules directly using the import keyword. This fails in pytest:

from citry_core._rust.template_parser import parse_tag
# ModuleNotFoundError: No module named 'citry_core._rust.template_parser'

The cause is likely because the Rust-generated submodules (like _rust.template_parser) are virtual modules - they don't exist as actual files, but only inside the Rust binary.

To fix the issue with pytest, we have to avoid importing the virtual Python modules using import keyword.

This simply means that, instead of importing the Rust API as:

from citry_core._rust.html_transform import transform_html

transform_html(...)

We have to do:

from citry_core import _rust

_rust.html_transform.transform_html(...)

Development

Build

Build the package using maturin:

cd packages/py/citry_core
maturin develop  # Development build
maturin build    # Production build

Or with uv:

uv run maturin develop

Test

Important: Before running tests, you must install the package in development mode:

cd packages/py/citry_core
uv run maturin develop

This builds the Rust extension module and installs the package so Python can import it. Without this step, tests will fail with ModuleNotFoundError: No module named 'citry_core'.

After installing, run tests:

# From the package directory
uv run pytest

Adding a new Rust crate

When a new Rust crate / submodule is added to the twin Rust crate citry_core_py:

1. Add to _rust.pyi: Define the type stubs

   class new_module:
       def new_function(...) -> ...:
           """Docstring"""
           ...
   

2. Create Python wrapper: Add citry_core/new_module/__init__.py if needed

NOTE: A validator bindings (run by python scripts/check.py) checks that we've added entries to _rust.pyi and that the new_module subdirectory exists.

Type stub maintenance

When updating Rust functions:

1. Update the function signature and docstring in _rust.pyi
2. Ensure the filename _rust.pyi matches the second part from module-name in pyproject.toml