paramclasses 0.3.0

Parameter-holding classes with robust subclassing protection

ParamClass

# Install from PyPI
pip install paramclasses

Table of Contents

1. 👩‍🏫 Rationale
2. 🧐 Overview
   • Defining a paramclass
   • Protecting attributes with @protected
   • Seamless attributes interactions
   • Expected getattr, setattr and delattr behaviour
   • Additional functionalities
     • Callback on parameters updates
     • Instantiation logic with __post_init__
     • Abstract methods
3. 👩‍💻 Subclassing API
4. 🤓 Advanced
   • Post-creation protection
   • Descriptor parameters
   • Multiple inheritance
   • @protected vs. super()
   • Using __slots__
   • Breaking ParamClass protection scheme
   • Type checkers
5. 🚀 Contributing
   • Developing with uv
6. ⚖️ License

1. Rationale 👩‍🏫

Parameter-holding classes vs. inheritance...

For a parameter-holding class, like dataclasses, it would be nice to embark some inherited functionality -- e.g. params property to access current (param, value) pairs, missing_params for unassigned parameter keys,... Such inheritance would allow to factor out specialized functionality for context-dependant methods -- e.g. fit, reset, plot, etc... However, such subclassing comes with a risk of attributes conflicts, especially for libraries or exposed APIs, when users do not necessarily know every "read-only" (or "protected") attributes from base classes.

Our solution 😌

To solve this problem, we propose a base ParamClass and an @protected decorator, which robustly protects any target attribute -- not only parameters -- from being accidentally overriden when subclassing, at runtime. If a subclass tries to override an attribute protected by one of its parents, a detailed ProtectedError will be raised and class definition will fail.

Why not use @dataclass(frozen=True) or typing.final?

First of all, the @dataclass(frozen=True) decorator only applies protection to instances. Besides, it targets all attributes indifferently. Morover, it does not protect against deletion or direct vars(instance) manipulation. Finally, protection is not inherited, thus subclasses need to use the decorator again, while being cautious not to silently override previously protected attributes.

The typing alternatives @final and Final are designed for type checkers on which we do not want to rely. From python 3.11 onwards, final does add a __final__ flag when possible, but it will not affect immutable objects.

We also mention this recent PEP draft considering attribute-level protection, again for type checkers and without considering subclassing protection.

Disclaimer

Note that the protection provided by paramclasses is very robust for practical use, but it should not be considered a security feature.

^{Back to Table of Contents👆}

2. Overview 🧐

Defining a paramclass

A paramclass is simply defined by subclassing ParamClass directly or another paramclass. Similarly to dataclasses, parameters are identified as any annotated attribute and instancation logic is automatically built-in -- though it can be extended.

from paramclasses import ParamClass

class A(ParamClass):
    parameter_with_a__default_value: ... = "default value"
    parameter_with_no_default_value: ...
    not_a_parameter = "not a parameter"
    def an_actual_method(self): ...
    def a_method_turned_into_a_parameter(self): ...
    a_method_turned_into_a_parameter: ...

Instances have a repr -- which can be overriden in subclasses -- displaying non-default or missing parameter values.

>>> A(parameter_with_a__default_value="non-default value")
A(parameter_with_a__default_value='non-default value', parameter_with_no_default_value=?)

One accesses current parameters dict and missing parameters of an instance with the properties params and missing_params respectively.

>>> from pprint import pprint
>>> pprint(A().params)
{'a_method_turned_into_a_parameter': <function A.a_method_turned_into_a_parameter at 0x11067b9a0>,
 'parameter_with_a__default_value': 'default value',
 'parameter_with_no_default_value': ?}
>>> A().missing_params
('parameter_with_no_default_value',)

Note that A().a_method_turned_into_a_parameter is not a bound method -- see Descriptor parameters.

^{Back to Table of Contents👆}

Protecting attributes with @protected

Say we define the following BaseEstimator class.

from paramclasses import ParamClass, protected

class BaseEstimator(ParamClass):
    @protected
    def fit(self, data): ...  # Some fitting logic

Then, we are guaranteed that no subclass can redefine fit.

>>> class Estimator(BaseEstimator):
...     fit = True  # This should FAIL
... 
<traceback>
ProtectedError: 'fit' is protected by 'BaseEstimator'

This runtime protection can be applied to all methods, properties, attributes -- with protected(value) --, etc... during class definition but not after. It is "robust" in the sense that breaking the designed behaviour, though possible, requires -- to our knowledge -- obscure patterns.

^{Back to Table of Contents👆}

Seamless attributes interactions

Parameters can be assigned values like any other attribute -- unless specifically protected -- with instance.attr = value. It is also possible to set multiple parameters at once with keyword arguments during instantiation, or after with set_params.

class A(ParamClass):
    x: ...      # Parameter without default value
    y: ... = 0  # Parameter with default value `0`
    z: ... = 0  # Parameter with default value `0`
    t = 0       # Non-parameter attribute

>>> a = A(y=1); a.t = 1; a    # Instantiation assignments
A(x=?, y=1)                   # Only shows missing and non-default parameters
>>> A().set_params(x=2, y=2)  # `set_params` assignments
>>> A().y = 1                 # Usual assignment
>>> del A(x=0).x              # Usual deletion
>>> A.y = 1                   # Class-level assignment/deletion works...
>>> A()
A(x=?, y=1)                   # ... and `A` remembers default values -- otherwise would show `A(x=?)`
>>> a.set_params(t=0)         # Should FAIL: Non-parameters cannot be assigned with `set_params`
<traceback>
AttributeError: Invalid parameters: {'t'}. Operation cancelled

^{Back to Table of Contents👆}

Expected getattr, setattr and delattr behaviour

Table of Expected Behaviour

Operation on Class or instance	Parameters	Non-Parameters
Protected	Unprotected	Protected	Unprotected
getattr	Bypass Descriptors*	Bypass Descriptors	Vanilla*	Vanilla
setattr	ProtectedError	Bypass Descriptors	ProtectedError	Vanilla
delattr	ProtectedError	Bypass Descriptors	ProtectedError	Vanilla

*_{On instance, getattr should ignore and remove any vars(instance) entry.}

Vanilla means that there should be no discernable difference compared to standard classes.

^{Back to Table of Contents👆}

Additional functionalities

Callback on parameters updates

Whenever an instance is assigned a value -- instantiation, set_params, dotted assignment -- the callback

def _on_param_will_be_set(self, attr: str, future_val: object) -> None

is triggered. For example, it can be used to unfit and estimator on specific modifications. As suggested by the name and signature, the callback operates just before the future_val assignment. There is currently no counterpart for parameter deletion. This could be added upon motivated interest.

^{Back to Table of Contents👆}

Instantiation logic with __post_init__

Similarly to dataclasses, a __post_init__ method can be defined to complete instantiation after the initial setting of parameter values. It must have signature

def __post_init__(self, *args: object, **kwargs: object) -> None

and is called as follows by __init__.

# Close equivalent to actual implementation
@protected
def __init__(self, args: list = [], kwargs: dict = {}, /, **param_values: object) -> None:
        self.set_params(**param_values)
        self.__post_init__(*args, **kwargs)

Since parameter values are set before __post_init__ is called, they are accessible when it executes.

^{Back to Table of Contents👆}

Abstract methods

The base ParamClass already inherits ABC functionalities, so @abstractmethod can be used.

from abc import abstractmethod

class A(ParamClass):
    @abstractmethod
    def next(self): ...

>>> A()
<traceback>
TypeError: Can't instantiate abstract class A with abstract method next

^{Back to Table of Contents👆}

3. Subclassing API 👩‍💻

As seen in Additional functionalities, three methods may be overriden by subclasses.

# ===================== Subclasses may override these ======================
def _on_param_will_be_set(self, attr: str, future_val: object) -> None:
    """Call before parameter assignment."""

def __post_init__(self, *args: object, **kwargs: object) -> None:
    """Init logic, after parameters assignment."""

def __repr__(self) -> str:
    """Show all non-default or missing, e.g. `A(x=1, z=?)`."""

Furthermore, as a last resort, developers may occasionally wish to use the following module attributes.

• IMPL: Current value is "__paramclass_impl_". Use getattr(paramclass or instance, IMPL) to get a NamedTuple instance with default and protected fields. Both are mapping proxies of, respectively, (param, default value) and (protected attribute, owner) pairs. The string IMPL acts as special protected key for paramclasses' namespaces, to leave default and protected available to users. We purposefully chose a would-be-mangled name to further decrease the odds of natural conflict.
• MISSING: The object representing the "missing value" in the default values of parameters. Using instance.missing_params should almost always be enough, but if necessary, use val is MISSING to check for missing values.

# Recommended way of using `IMPL`
from paramclasses import IMPL, ParamClass

getattr(ParamClass, IMPL).default    # mappingproxy({})
getattr(ParamClass, IMPL).protected  # mappingproxy({'__paramclass_impl_': None, '__dict__': None, '__init__': <class 'paramclasses.paramclasses.RawParamClass'>, '__getattribute__': <class 'paramclasses.paramclasses.RawParamClass'>, '__setattr__': <class 'paramclasses.paramclasses.RawParamClass'>, '__delattr__': <class 'paramclasses.paramclasses.RawParamClass'>, 'set_params': <class 'paramclasses.paramclasses.ParamClass'>, 'params': <class 'paramclasses.paramclasses.ParamClass'>, 'missing_params': <class 'paramclasses.paramclasses.ParamClass'>})
# Works on subclasses and instances too

When subclassing an external UnknownClass, one can check whether it is a paramclass with isparamclass.

from paramclasses import isparamclass

isparamclass(UnknownClass)  # Returns a boolean

Finally, it is possible to subclass RawParamClass directly -- unique parent class of ParamClass --, when set_params, params and missing_params are not necessary. In this case, use signature isparamclass(UnknownClass, raw=True).

^{Back to Table of Contents👆}

4. Advanced 🤓

Post-creation protection

It is not allowed and will be ignored with a warning.

class A(ParamClass):
    x: int = 1

>>> A.x = protected(2)  # Assignment should WORK, protection should FAIL
<stdin>:1: UserWarning: Cannot protect attribute 'x' after class creation. Ignored
>>> a = A(); a
A(x=2)                  # Assignment did work
>>> a.x = protected(3)  # Assignment should WORK, protection should FAIL
<stdin>:1: UserWarning: Cannot protect attribute 'x' on instance assignment. Ignored
>>> a.x
3                       # First protection did fail, new assignment did work
>>> del a.x; a
A(x=2)                  # Second protection did fail

^{Back to Table of Contents👆}

Descriptor parameters

TLDR: using descriptors for parameter values is fine if you know what to expect.

import numpy as np

class Operator(ParamClass):
    op: ... = np.cumsum

Operator().op([0, 1, 2])  # array([0, 1, 3])

This behaviour is similar to dataclasses' but is not trivial:

class NonParamOperator:
    op: ... = np.cumsum

>>> NonParamOperator().op([0, 1, 2])  # Should FAIL
<traceback>
TypeError: 'list' object cannot be interpreted as an integer
>>> NonParamOperator().op
<bound method cumsum of <__main__.NonParamOperator object at 0x13a10e7a0>>

Note how NonParamOperator().op is a bound method. What happened here is that since np.cumsum is a data descriptor -- like all function, property or member_descriptor objects for example --, the function np.cumsum(a, axis=None, dtype=None, out=None) interpreted NonParamOperator() to be the array a, and [0, 1, 2] to be the axis.

To avoid this kind of surprises we chose, for parameters only, to bypass the get/set/delete descriptor-specific behaviours, and treat them as usual attributes. Contrary to dataclasses, by also bypassing descriptors for set/delete operations, we allow property-valued parameters, for example.

class A(ParamClass):
    x: property = property(lambda _: ...)  # Should WORK

@dataclass
class B:
    x: property = property(lambda _: ...)  # Should FAIL

>>> A()  # paramclass
A()
>>> B()  # dataclass
<traceback>
AttributeError: can't set attribute 'x'

This should not be a very common use case anyway.

^{Back to Table of Contents👆}

Multiple inheritance

With paramclass bases

Multiple inheritance is not a problem. Default values will be retrieved as expect following the MRO, but there's one caveat: protected attributes should be consistant between the bases. For example, if A.x is not protected while B.x is, one cannot take (A, B) for bases.

class A(ParamClass):
    x: int = 0

class B(ParamClass):
    x: int = protected(1)

class C(B, A): ...  # Should WORK

class D(A, B): ...  # Should FAIL

>>> class C(B, A): ...  # Should WORK
... 
>>> class D(A, B): ...  # Should FAIL
... 
<traceback>
ProtectedError: 'x' protection conflict: 'A', 'B'

Inheriting from non-paramclasses

It is possible to inherit from a mix of paramclasses and non-paramclasses, with the two following limitations.

1. Because type(ParamClass) is a subclass of ABCMeta, non-paramclass bases must be either vanilla classes or abstract classes.

2. Behaviour is not guaranteed for non-paramclass bases with an IMPL-named attribute -- see Subclassing API.

^{Back to Table of Contents👆}

@protected vs. super()

It is not recommended to use super() inside a @protected method definition, when the protection aims at "locking down" its behaviour. Indeed, one can never assume the MRO of future subclasses will ressemble that of the method-defining class.

For example, picture the following inheritance schemes.

class A(RawParamClass): ...
class B(RawParamClass): ...
class C(B, A): ...

In this situation, the MRO of C would be C -> B -> A -> RawParamClass -> object. As such, if B was to redefine __repr__ using super() and @protected, repr(C()) would call A.__repr__, which can behave arbitrarily despite B.__repr__ being @protected. Instead, it is recommended to call RawParamClass.__repr__ directly.

^{Back to Table of Contents👆}

Using __slots__

Before using __slots__ with ParamClass, please note the following.

1. Currently paramclasses do not use __slots__, so any of its subclasses will still have a __dict__. More on that in the future...
2. You cannot slot a previously protected attribute -- since it would require replacing its value with a member object.
3. Since parameters' get/set/delete interactions bypass descriptors, using __slots__ on them will not yield the usual behaviour.
4. The overhead from ParamClass functionality, although not high, probably nullifies any __slots__ optimization in most use cases.

^{Back to Table of Contents👆}

Breaking ParamClass protection scheme

There is no such thing as "perfect attribute protection" in Python. As such ParamClass only provides protection against natural behaviour -- and even unnatural to a large extent. Below are some knonwn anti-patterns to break it, representing discouraged behaviour. If you find other elementary ways, please report them in an issue.

1. Using type.__setattr__/type.__delattr__ directly on paramclasses.
2. Modifying @protected -- huh?
3. Modifying or subclassing type(ParamClass) -- requires evil dedication.
4. Messing with mappingproxy, which is not really immutable.

^{Back to Table of Contents👆}

Type checkers

The @protected decorator is not acting in the usual sense, as it is a simple wrapper meant to be detected and unwrapped by the metaclass constructing paramclasses. Consequently, type checkers such as mypy may be confused. If necessary, we recommend locally disabling type checking with the following comment -- and the appropriate error-code.

@protected  # type: ignore[error-code]  # mypy is fooled
def my_protected_method(self):

It is not ideal and may be fixed in future updates.

^{Back to Table of Contents👆}

5. Contributing 🚀

Questions, issues, discussions and pull requests are welcome! Please do not hesitate to contact me.

Developing with uv

The project is developed with uv which simplifies soooo many things!

# Installing `uv` on Linux and macOS
curl -LsSf https://astral.sh/uv/install.sh | sh
# Using `uv` command may require restarting the bash session

After having installed uv, you can independently use all of the following without ever worrying about installing python or dependencies, or creating virtual environments.

uvx ruff check                        # Check linting
uvx ruff format --diff                # Check formatting
uv run mypy                           # Run mypy
uv pip install -e . && uv run pytest  # Run pytest
uv run python                         # Interactive session in virtual environment

^{Back to Table of Contents👆}

6. License ⚖️

This package is distributed under the MIT License.

^{Back to Table of Contents👆}