stubgen-pyx

Generate Python stub files (.pyi) from Cython source code (.pyx/.pxd)

Automatic stub file generation for Cython extensions that enables full IDE support and type checking for Cython modules.

Used By

• NVIDIA
• cyndilib

Are you using this project in production or in an open-source tool? We'd love to feature you! Please open a pull request to add your project, company, or application to this list.

Table of Contents

• Features
• Installation
• Quick Start
• Usage
• How It Works
• Supported Cython Features
• Configuration Options
• Development
• Contributing

Features

Comprehensive Cython Support

• Extracts type information from Cython source files
• Preserves docstrings (if specified) and function signatures
• Handles both .pyx and .pxd files
• Supports class hierarchies and inheritance

Smart Processing

• Automatic import trimming and deduplication
• Cython type normalization (e.g., bint → bool, unicode → str)
• Undefined name elimination from type hints and defaults
• Proper handling of positional-only and keyword-only arguments
• Preserves decorators and class metadata

Installation

pip install stubgen-pyx

Quick Start

Generate stubs for all Cython files in the current directory:

stubgen-pyx .

Or use the Python API:

from stubgen_pyx import StubgenPyx
from pathlib import Path

stubgen = StubgenPyx()
results = stubgen.convert_glob("**/*.pyx")

for result in results:
    if result.success:
        print(f"Generated: {result.pyi_file}")
    else:
        print(f"Failed: {result.pyx_file}")
        print(f"  Error: {result.error}")

Usage

Command Line

Basic usage:

# Convert all .pyx files in current directory
stubgen-pyx .

# Convert files in a specific directory
stubgen-pyx /path/to/cython/package

# Use custom glob pattern
stubgen-pyx . --file "**/*.pyx"

Advanced options:

# Preview changes without writing
stubgen-pyx . --dry-run

# Enable verbose logging
stubgen-pyx . --verbose

# Continue processing even if some files fail
stubgen-pyx . --continue-on-error

Output options:

# Note: --output-dir and --output-file are mutually exclusive

# Write all .pyi files to a custom directory
stubgen-pyx . --output-dir stubs/

# Convert a single .pyx file to a specific output path
# (requires exactly one matching input file)
stubgen-pyx . --file mymodule.pyx --output-file output/mymodule.pyi

# Exclude certain files from conversion
stubgen-pyx . --exclude-pattern "**/tests/**" --exclude-pattern "vendor/**"

Disable specific transformations:

# Disable import sorting
stubgen-pyx . --no-sort-imports

# Don't trim unused imports
stubgen-pyx . --no-trim-imports

# Don't normalize Cython types (keep bint, unicode, etc.)
stubgen-pyx . --no-normalize-names

# Exclude .pxd file contents
stubgen-pyx . --no-pxd-to-stubs

# Skip deduplicating imports
stubgen-pyx . --no-deduplicate-imports

# Skip trimming undefined names from annotations and defaults
stubgen-pyx . --no-trim-not-defined

# Skip preserving docstrings from the stub
stubgen-pyx . --exclude-docstrings

# Skip adding stubgen-pyx attribution comment
stubgen-pyx . --exclude-attribution

# Include private functions in the stub
stubgen-pyx . --include-private

Python API

Basic conversion:

from stubgen_pyx import StubgenPyx
from pathlib import Path

stubgen = StubgenPyx()

# Convert a single file
pyx_code = Path("mymodule.pyx").read_text()
pyi_stub = stubgen.convert_str(pyx_code)
print(pyi_stub)

Batch processing with results:

from stubgen_pyx import StubgenPyx

stubgen = StubgenPyx()
results = stubgen.convert_glob("src/**/*.pyx")

successful = sum(1 for r in results if r.success)
failed = sum(1 for r in results if not r.success)

print(f"Converted: {successful}/{len(results)} files")
if failed > 0:
    for result in results:
        if not result.success:
            print(f"  - {result.pyx_file}: {result.error}")

Custom configuration:

from stubgen_pyx import StubgenPyx
from stubgen_pyx.config import StubgenPyxConfig

config = StubgenPyxConfig(
    trim_imports=False,             # Don't trim unused imports
    normalize_names=False,          # Don't normalize Cython types
    sort_imports=False,             # Don't sort imports
    deduplicate_imports=False,      # Don't deduplicate imports
    exclude_attribution=False,      # Don't skip stubgen-pyx attribution comment
    continue_on_error=True,         # Continue on errors
    include_private=False,          # Exclude private functions
    verbose=True,                   # Show details
)

stubgen = StubgenPyx(config=config)
results = stubgen.convert_glob("**/*.pyx")

How It Works

stubgen-pyx works in several stages:

1. Parsing: Cython source code is parsed using the Cython compiler's AST
2. Analysis: The AST is visited to extract type information, signatures, and docstrings
3. Conversion: Cython-specific constructs are converted to intermediate PyiElements
4. Building: PyiElements are transformed into Python stub code
5. Postprocessing: Generated code is optimized (imports trimmed, types normalized, etc.)

Why stubgen-pyx vs mypy's stubgen?

While mypy's stubgen can generate stubs for compiled extension modules through runtime introspection, it cannot access Cython-specific metadata embedded in the source code. This results in:

• Missing type annotations
• Incomplete function signatures
• No support for Cython-specific constructs (cdef classes, memory views)

stubgen-pyx directly analyzes the Cython source, providing:

• Complete type information
• Accurate function signatures with annotations
• Preserved docstrings and decorators

Supported Cython Features

Supported

• Python functions (def)
• C functions (cdef)
• C/Python functions (cpdef)
• Classes (both Python class and Cython cdef class)
• Class inheritance and metaclasses
• Type annotations on arguments and return values
• Docstrings
• Decorators
• Default arguments
• *args and **kwargs
• Keyword-only arguments
• Positional-only arguments
• Cython enums (cdef enum)
• Import statements (including cimport)
• Public attributes and properties
• Typed memoryviews (e.g. double[:, :] → numpy.typing.NDArray[numpy.double])
• Fixed-size C arrays (e.g. char[100] → bytes, int[100][100] → list[list[int]])
• Pointer-to-char declarations (char * → bytes)
• Fused types (e.g. FooOrBar → TypeVar('FooOrBar', Foo, Bar) or Foo | Bar as appropriate)

Configuration Options

All configuration is handled through the StubgenPyxConfig dataclass:

Example

Input: Cython module

# signal_processing.pyx
"""Signal processing utilities for scientific computing."""

cdef class Filter:
    """A digital signal filter."""

    cdef public char *name
    cdef public int order
    cdef double[32] _coeffs

    def __init__(self, char *name, int order, user_data: tuple[str, ...] = ()):
        """Initialize the filter."""
        self.name = name
        self.order = order

    cpdef double[:] apply(self, double[:] signal):
        """Apply the filter to a 1-D signal."""
        pass

    cpdef double[:, :] apply_batch(self, double[:, :] signals):
        """Apply the filter to a batch of signals (one per row)."""
        pass

    cdef int _validate_coeffs(self):
        """Internal coefficient validation."""
        return 0

Output: Generated stub

# signal_processing.pyi

# This file was generated by stubgen-pyx v0.x.x from signal_processing.pyx
"""Signal processing utilities for scientific computing."""
import numpy


class Filter:
    """A digital signal filter."""
    name: bytes
    order: int

    def __init__(self, name: bytes, order: int, user_data: tuple[str, ...]=()):
        """Initialize the filter."""
    def apply(self, signal: numpy.typing.NDArray[numpy.double]) -> numpy.typing.NDArray[numpy.double]:
        """Apply the filter to a 1-D signal."""
    def apply_batch(self, signals: numpy.typing.NDArray[numpy.double]) -> numpy.typing.NDArray[numpy.double]:
        """Apply the filter to a batch of signals (one per row)."""

Note:

• Public methods and attributes are included; private ones (_validate_coeffs, _coeffs) are excluded
• char * and char[N] declarations map to bytes
• Typed memoryviews map to numpy.typing.NDArray[dtype]
• Cdef functions without Python wrappers are excluded from the stub
• Docstrings and default argument values are preserved

Development

Setup

# Clone the repository
git clone https://github.com/jon-edward/stubgen-pyx.git
cd stubgen-pyx

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install in development mode with test dependencies
pip install -e ".[test]"

Running Tests

# Run all tests
pytest

# Run specific test file
pytest tests/test_config.py

Project Structure

stubgen-pyx/
├── stubgen_pyx/              # Main package
│   ├── analysis/             # AST analysis (visitors)
│   ├── builders/             # Code generation
│   ├── conversion/           # AST conversion
│   ├── models/               # Data models (PyiElements)
│   ├── parsing/              # Cython parser
│   ├── postprocessing/       # Output optimization
│   ├── cli.py                # Command-line interface
│   ├── config.py             # Configuration
│   └── stubgen.py            # Main entry point
├── tests/                    # Test suite
├── README.md                 # This file
└── pyproject.toml            # Project metadata

Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Write tests for new functionality
4. Ensure all tests pass (pytest)
5. Commit your changes (git commit -m 'Add amazing feature')
6. Push to your fork (git push origin feature/amazing-feature)
7. Open a Pull Request

License

See LICENSE file for details.

Acknowledgments

• Built on top of the Cython compiler infrastructure
• Inspired by mypy's stubgen