py-meta-utils 0.8.0

Metaclass and metaprogramming utilities for Python

Py Meta Utils

Useful Links

• Official Documentation on Read The Docs
• Source Code on GitHub
• PyPI

The Meta Options Factory Pattern as a library, and related metaclass utilities

When you as a library or framework author want your end users to be able to write something like this:

class SomeClass(YourBaseClass):
    class Meta:
        option1 = 'value1'
        option2 = 'value2'
        option3 = 'value3'

And you need a way to define each of these option/value pairs, and a way to "attach" custom behavior to them (ie, code that manipulates code using a custom metaclass on YourBaseClass). There are a couple common-ish patterns to accomplish this. Django and Graphene have one way, Marshmallow another, and Factory Boy another (and no doubt probably others). But Factory Boy's implementation is by far the most powerful and flexible one I've come across.

I discovered this pattern while reading the source code of factory_boy (specifically, this file).

And I decided to extract it and turn it into a reusable library. In the process, I ended up refactoring a few things and adding a couple niceties to improve upon its usage.

Let's take a look at a silly example to allow your end users to be able to optionally enable logging of the actions of a class from a library you're writing:

class EndUserClass(YourLoggableService):
    class Meta:
        debug: bool = True
        verbosity: int = 2
        log_destination: str = '/tmp/end-user-class.log'

The first step is to define your custom MetaOption subclasses:

• All that's absolutely required to implement is the constructor and its name argument. That said, it's recommended to also specify the default and inherit arguments for the sake of being explicit.
• The check_value method is optional, but useful for making sure your users aren't giving you garbage.
• The get_value method has a default implementation that normally you shouldn't need to override, unless your default value is mutable or you have advanced logic.
• There's also a contribute_to_class method that we'll cover later on.

import os
import sys

# first we have to import what we need from py_meta_utils
from py_meta_utils import (McsArgs, MetaOption, MetaOptionsFactory,
                           process_factory_meta_options, _missing)

# then we have to declare the meta options the meta options factory should support
class DebugMetaOption(MetaOption):
    def __init__(self):
        super().__init__(name='debug', default=False, inherit=True)

    def check_value(self, value, mcs_args: McsArgs):
        if not isinstance(value, bool):
            raise TypeError(f'The {self.name} Meta option must be a bool')


class VerbosityMetaOption(MetaOption):
    def __init__(self):
        super().__init__(name='verbosity', default=1, inherit=True)

    def check_value(self, value, mcs_args: McsArgs):
        if value not in {1, 2, 3}:
            raise ValueError(f'The {self.name} Meta option must either 1, 2, or 3')


class LogDestinationMetaOption(MetaOption):
    def __init__(self):
        super().__init__(name='log_destination', default=_missing, inherit=True)

    # this pattern is useful if you need a mutable default value like [] or {}
    def get_value(self, Meta, base_classes_meta, mcs_args: McsArgs):
        value = super().get_value(Meta, base_classes_meta, mcs_args)
        return value if value != _missing else 'stdout'

    def check_value(self, value, mcs_args: McsArgs):
        if value in {'stdout', 'stderr'}:
            return

        try:
            dir_exists = os.path.exists(os.path.dirname(value))
        except:
            dir_exists = False

        if not dir_exists:
            raise ValueError(f'The {self.name} Meta option must be one of `stdout`, '
                             '`stderr`, or a valid filepath')

The next step is to subclass MetaOptionsFactory and specify the MetaOption subclasses you want:

class LoggingMetaOptionsFactory(MetaOptionsFactory):
    _options = [
        DebugMetaOption,
        VerbosityMetaOption,
        LogDestinationMetaOption,
    ]

Then you need a metaclass to actually apply the factory options:

class LoggingMetaclass(type):
    def __new__(mcs, name, bases, clsdict):
        mcs_args = McsArgs(mcs, name, bases, clsdict)
        process_factory_meta_options(mcs_args, LoggingMetaOptionsFactory)
        return super().__new__(*mcs_args)

And lastly, create the public class, using the metaclass just defined:

class YourLoggableService(metaclass=LoggingMetaclass):
    def do_important_stuff(self):
        if self.Meta.verbosity < 2:
            self._log('doing important stuff')
        else:
            self._log('doing really detailed important stuff like so')

    def _log(self, msg):
        if not self.Meta.debug:
            return

        if self.Meta.log_destination == 'stdout':
            print(msg)
        elif self.Meta.log_destination == 'stderr':
            sys.stderr.write(msg)
            sys.stderr.flush()
        elif self.Meta.log_destination:
            with open(self.Meta.log_destination, 'a') as f:
                f.write(msg)

The options factory automatically adds the Meta attribute to the class-under-construction (in this example, YourLoggableService). (In this case the Meta attribute will be populated with the default values as supplied by the MetaOption subclasses specified by the factory.) In the case where the class-under-construction has a partial Meta class, the missing meta options will be added to it.(*)

(*) In effect that's what happens, and for all practical purposes is probably how you should think about it, but technically speaking, the class-under-construction's Meta attribute actually gets replaced with a populated instance of the specified MetaOptionsFactory subclass.

The one thing we didn't cover is MetaOption.contribute_to_class. This is an optional callback hook that allows MetaOption subclasses to, well, contribute something to the class-under-construction. Most likely it adds/removes attributes to/from the class, or perhaps it wraps some method(s) with a decorator or something else entirely.

A good simple example can be found in the source code for the included AbstractMetaOption:

ABSTRACT_ATTR = '__abstract__'


class AbstractMetaOption(MetaOption):
    def __init__(self):
        super().__init__(name='abstract', default=False, inherit=False)

    def get_value(self, Meta, base_classes_meta, mcs_args: McsArgs):
        # class attributes take precedence over the class Meta's value
        if mcs_args.clsdict.get(ABSTRACT_ATTR, False) is True:
            return True
        return super().get_value(Meta, base_classes_meta, mcs_args) is True

    def contribute_to_class(self, mcs_args: McsArgs, value):
        if value is True:
            mcs_args.clsdict[ABSTRACT_ATTR] = True
        else:
            mcs_args.clsdict[ABSTRACT_ATTR] = False

A number of libraries use the __abstract__ class attribute to determine whether or not the class-under-construction should be considered concrete or not, but they won't understand class Meta options. Therefore, we implement MetaOption.contribute_to_class to set the __abstract__ class attribute to the appropriate value for backwards compatibility with such libraries.

Included Metaclass Utilities

Singleton

Singleton is an included metaclass that makes any class utilizing it a singleton:

from py_meta_utils import Singleton


class YourSingleton(metaclass=Singleton):
    pass


instance = YourSingleton()
assert instance == YourSingleton()

Classes using Singleton can be subclassed, however, you must inform the base class of your subclass:

from py_meta_utils import Singleton

class BaseSingleton(metaclass=Singleton):
    pass

class Extended(BaseSingleton):
    pass

BaseSingleton.set_singleton_class(Extended)
base_instance = BaseSingleton()
extended_instance = Extended()
assert base_instance == extended_instance == BaseSingleton() == Extended()

deep_getattr

deep_getattr(clsdict, bases, 'attr_name', [default])

deep_getattr acts just like getattr would on a constructed class object, except this operates on the pre-class-construction class dictionary and base classes. In other words, first we look for the attribute in the class dictionary, and then we search all the base classes (in method resolution order), finally returning the default value if the attribute was not found in any of the class dictionary or base classes (or it raises AttributeError if default not given).

OptionalMetaclass and OptionalClass

try:
    from optional_dependency import SomeClass
except ImportError:
    from py_meta_utils import OptionalClass as SomeClass


class Optional(SomeClass):
    pass

License

MIT