Assorted function decorators.

Short summary:

• ALL: Include this function's name in its module's __all__ list.

• attr: A decorator to set attributes on a function. Example:.

• cached: OBSOLETE version of cached, suggestion: cs.cache.cachedmethod.

• cachedmethod: OBSOLETE version of cachedmethod, suggestion: cs.cache.cachedmethod.

• contextdecorator: A decorator for a context manager function cmgrfunc which turns it into a decorator for other functions.

• contextual: Wrap a simple function as a context manager.

• decorator: A decorator for decorator functions to support optional arguments and also to allow specifying additional attributes to apply to the decorated function.

• default_params: A decorator to provide factory functions for default parameters.

• fmtdoc: A decorator to format a function's docstring. This replaces the docstring with that string formatted against the function's module __dict__ using str.format_map.

• logging_wrapper: Decorator for logging call shims which bumps the stacklevel keyword argument so that the logging system chooses the correct frame to cite in messages.

• observable_class: Class decorator to make various instance attributes observable.

• OBSOLETE: A decorator for obsolete functions or classes.

• Promotable: A mixin class which supports the @promote decorator by providing a default .promote(cls,obj) class method.

• promote: A decorator to promote argument values automatically in annotated functions.

• strable: Decorator for functions which may accept a str instead of their core type.

• uses_cmd_options: A decorator to provide default keyword arguments from the prevailing cs.cmdutils.BaseCommandOptions if available, otherwise from option_defaults.

• verbose: A decorator to run func if the ambient verbosity is >=1.

• verbosity: A decorator to run func if the ambient verbosity is >= verbosity_level.

• vv: A decorator to run func if the ambient verbosity is >=2.

• with_: A decorator to run func inside a with obj:.

Functions

ALL(func)

Include this function's name in its module's __all__ list.

Example:

from cs.deco import ALL

__all__ = []

def obscure_function(...):
    ...

@ALL
def well_known_function(...):
    ...

OBSOLETE(*da, **dkw)

A decorator for obsolete functions or classes.

Use:

@OBSOLETE
def func(...):

or

@OBSOLETE("new_func_name")
def func(...):

This emits a warning log message before calling the decorated function. Only one warning is emitted per calling location.

It also marks the function as deprecated using @warnings.deprecated.

attr(*da, **dkw)

A decorator to set attributes on a function. Example:

@attr(hook_names=('hook1', 'hook2'))
def func():
    .....

This is just a more overt and clear form of:

def func():
    .....

func.hook_names=('hook1', 'hook2')

cached(*a, **kw)

OBSOLETE version of cached, suggestion: cs.cache.cachedmethod

Former name for @cachedmethod.

cachedmethod(*a, **kw)

OBSOLETE version of cachedmethod, suggestion: cs.cache.cachedmethod

@cachedmethod is now in cs.cache.

contextdecorator(*da, **dkw)

A decorator for a context manager function cmgrfunc which turns it into a decorator for other functions.

This supports easy implementation of "setup" and "teardown" code around other functions without the tedium of defining the wrapper function itself. See the examples below.

The resulting context manager accepts an optional keyword parameter provide_context, default False. If true, the context returned from the context manager is provided as the first argument to the call to the wrapped function.

Note that the context manager function cmgrfunc has not yet been wrapped with @contextmanager, that is done by @contextdecorator.

This decorator supports both normal functions and generator functions.

With a normal function the process is:

• call the context manager with (func,a,kw,*da,**dkw), returning ctxt, where da and dkw are the positional and keyword parameters supplied when the decorator was defined.
• within the context return the value of func(ctxt,*a,**kw) if provide_context is true or the value of func(*a,**kw) if not (the default)

With a generator function the process is:

• obtain an iterator by calling func(*a,**kw)
• for iterate over the iterator, yielding its results, by calling the context manager with (func,a,kw,**da,**dkw), around each next() Note that it is an error to provide a true value for provide_context if the decorated function is a generator function.

Some examples follow.

Trace the call and return of a specific function:

@contextdecorator
def tracecall(func, a, kw):
    """ Trace the call and return from some function.
        This can easily be adapted to purposes such as timing a
        function call or logging use.
    """
    print("call %s(*%r,**%r)" % (func, a, kw))
    try:
      yield
    except Exception as e:
      print("exception from %s(*%r,**%r): %s" % (func, a, kw, e))
      raise
    else:
      print("return from %s(*%r,**%r)" % (func, a, kw))

@tracecall
def f():
    """ Some function to trace.
    """

@tracecall(provide_context=True):
def f(ctxt, *a, **kw):
    """ A function expecting the context object as its first argument,
        ahead of whatever other arguments it would normally require.
    """

See who is making use of a generator's values, when a generator might be invoked in one place and consumed elsewhere:

from cs.py.stack import caller

@contextdecorator
def genuser(genfunc, *a, **kw):
    user = caller(-4)
    print(f"iterate over {genfunc}(*{a!r},**{kw!r}) from {user}")
    yield

@genuser
def linesof(filename):
    with open(filename) as f:
        yield from f

# obtain a generator of lines here
lines = linesof(__file__)

# perhaps much later, or in another function
for lineno, line in enumerate(lines, 1):
    print("line %d: %d words" % (lineno, len(line.split())))

Turn on "verbose mode" around a particular function:

import sys
import threading
from cs.context import stackattrs

class State(threading.local):
    def __init__(self):
        # verbose if stderr is on a terminal
        self.verbose = sys.stderr.isatty()

# per thread global state
state = State()

@contextdecorator
def verbose(func):
    with stackattrs(state, verbose=True) as old_attrs:
        if not old_attrs['verbose']:
            print(f"enabled verbose={state.verbose} for function {func}")
        # yield the previous verbosity as the context
        yield old_attrs['verbose']

# turn on verbose mode
@verbose
def func(x, y):
    if state.verbose:
        # print if verbose
        print("x =", x, "y =", y)

# turn on verbose mode and also pass in the previous state
# as the first argument
@verbose(provide_context=True):
def func2(old_verbose, x, y):
    if state.verbose:
        # print if verbose
        print("old_verbosity =", old_verbose, "x =", x, "y =", y)

contextual(func)

Wrap a simple function as a context manager.

This was written to support users of @strable, which requires its open_func to return a context manager; this turns an arbitrary function into a context manager.

Example promoting a trivial function:

>>> f = lambda: 3
>>> cf = contextual(f)
>>> with cf() as x: print(x)
3

decorator(deco=None, *, decorable=)

A decorator for decorator functions to support optional arguments and also to allow specifying additional attributes to apply to the decorated function.

The decorated function is updated by functools.update_wrapper and also has its __name__ and __doc__ set to those from before the decoration.

The return of such a decorator is usually the conventional decorated function, but may alternatively be a (decorated,attrs) 2-tuple where attrs is a mapping of additional attributes to apply to decorated. This allows overriding the restored __name__ and __doc__ from above, and potentially other useful attributes.

The actual decorator function (eg mydeco) ends up being called as:

mydeco(func, *da, **dkw)

allowing da and dkw to affect the behaviour of the decorator mydeco.

@decorator itself accepts an optional decorable= keyword parameter; this is used as the test of the first positional argument to the subdecorator to see if it is the function it decorates. This defaults to callable, but may be specified; I have a decorator which can accept classes as positional arguments and classes are callable so I decorate it like this:

@decorator(decorable=lambda f: callable(f) and not isinstance(f, type))

so as not to mistake a class as the function being decoration.

Examples:

# define your decorator as if always called with func and args
@decorator
def mydeco(func, *da, arg2=None):
  ... decorate func subject to the values of da and arg2

# @mydeco called with defaults
@mydeco
def func1(...):
  ...

# @mydeco called with nondefault arguments
@mydeco('foo', arg2='bah')
def func2(...):
  ...

The @mydeco decorator itself is then written as though the arguments were always supplied.

default_params(*da, **dkw)

A decorator to provide factory functions for default parameters.

This decorator accepts the following special keyword parameter:

• _strict: default False; if true only replace genuinely missing parameters; if false also replace the traditional None placeholder value The remaining keyword parameters are factory functions providing the respective default values.

Typical use as a decorator factory:

# in your support module
uses_ds3 = default_params(ds3client=get_ds3client)

# calling code which needs a ds3client
@uses_ds3
def do_something(.., *, ds3client, ...):
    ... make queries using ds3client ...

This replaces the standard boilerplate and avoids replicating knowledge of the default factory as exhibited in this legacy code:

def do_something(.., *, ds3client=None, ...):
    if ds3client is None:
        ds3client = get_ds3client()
    ... make queries using ds3client ...

It's quite common for me to associate one of these with a class. I have a HasThreadState mixin class which maintains a thread-local state object which I use to store a per-thread "ambient" instance of the class so that it does not need to be plumbed through every call (including all the intermediate calls which have no interest in the object, the horror!) So I'll often do this:

class Thing(..., HasThreadState):
    ....
uses_thing = default_params(thing=Thing.default)

This can be used to provide the ambient instance of Thing to functions while allowing the caller to omit any mention of a Thing or to pass a specific instance if sensible. (In this examplke, Thing.default is a method provided by the mixin.)

And then there's the atypical one off direct use, which is not really a big win over the conventional way:

@default_params(dbconn=open_default_dbconn,debug=lambda:settings.DB_DEBUG_MODE)
def dbquery(query, *, dbconn):
    dbconn.query(query)

fmtdoc(func)

A decorator to format a function's docstring. This replaces the docstring with that string formatted against the function's module __dict__ using str.format_map.

A quirk of format_map allows us to also support:

• {name=}: the f-string {name=} notation
• {name==}: the f-string {name=} notation with the function module name prefixed

This supports simple formatted docstrings. Example:

FUNC_ENVVAR = 'FUNC_SETTING
FUNC_DEFAUlT = 12

@fmtdoc
def func():
    """
    Do something with the environment variable `${FUNC_ENVVAR}`.
    The default if no `${FUNC_ENVVAR}` comes from `{FUNC_DEFAUlT==}`.
    """
    print(os.environ.get(FUNC_ENVVAR, FUNC_DEFAUlT))

This gives func this docstring:

Do something with the environment variable `$FUNC_SETTING`.
The default if no `$FUNC_SETTING` comes from `module.func.FUNC_DEFAUlT=12`.

Warning: this decorator is intended for wiring "constants" into docstrings, not for dynamic values. Use for other types of values should be considered with trepidation.

logging_wrapper(*da, **dkw)

Decorator for logging call shims which bumps the stacklevel keyword argument so that the logging system chooses the correct frame to cite in messages.

Note: has no effect on Python < 3.8 because stacklevel only appeared in that version.

observable_class(property_names, only_unequal=False)

Class decorator to make various instance attributes observable.

Parameters:

• property_names: an interable of instance property names to set up as observable properties. As a special case a single str can be supplied if only one attribute is to be observed.
• only_unequal: only call the observers if the new property value is not equal to the previous proerty value. This requires property values to be comparable for inequality. Default: False, meaning that all updates will be reported.

promote(*da, **dkw)

A decorator to promote argument values automatically in annotated functions.

If the annotation is Optional[some_type] or Union[some_type,None] then the promotion will be to some_type but a value of None will be passed through unchanged.

The decorator accepts optional parameters:

• params: if supplied, only parameters in this list will be promoted
• types: if supplied, only types in this list will be considered for promotion

For any parameter with a type annotation, if that type has a .promote(value) class method and the function is called with a value not of the type of the annotation, the .promote method will be called to promote the value to the expected type.

Note that the Promotable mixin provides a .promote() method which promotes obj to the class if the class has a factory class method from_typename(obj) where typename is obj.__class__.__name__. A common case for me is lexical objects which have a from_str(str) factory to produce an instance from its textual form.

Additionally, if the .promote(value) class method raises a TypeError and value has a .as_typename attribute (where typename is the name of the type annotation), if that attribute is an instance method of value then promotion will be attempted by calling value.as_typename() otherwise the attribute will be used directly on the presumption that it is a property.

A typical promote(cls, obj) method looks like this:

@classmethod
def promote(cls, obj):
    if isinstance(obj, cls):
        return obj
    ... recognise various types ...
    ... and return a suitable instance of cls ...
    return super().promote(obj)

The above example is for subclasses of Promotable, which has a default promote() class method. In another class the super() call would be replace with raising a TypeError.

Example:

>>> from cs.timeseries import Epoch
>>> from typeguard import typechecked
>>>
>>> @promote
... @typechecked
... def f(data, epoch:Epoch=None):
...     print("epoch =", type(epoch), epoch)
...
>>> f([1,2,3], epoch=12.0)
epoch = <class 'cs.timeseries.Epoch'> Epoch(start=0, step=12)

Example using a class with an as_P instance method:

>>> class P:
...   def __init__(self, x):
...     self.x = x
...   @classmethod
...   def promote(cls, obj):
...     raise TypeError("dummy promote method")
...
>>> class C:
...   def __init__(self, n):
...     self.n = n
...   def as_P(self):
...     return P(self.n + 1)
...
>>> @promote
... def p(p: P):
...   print("P =", type(p), p.x)
...
>>> c = C(1)
>>> p(c)
P = <class 'cs.deco.P'> 2

Note: one issue with this is due to the conflict in name between this decorator and the method it looks for in a class. The promote method must appear after any methods in the class which are decorated with @promote, otherwise the promote method supplants the name promote making it unavailable as the decorator. I usually just make .promote the last method.

Failing example:

class Foo:
    @classmethod
    def promote(cls, obj):
        ... return promoted obj ...
    @promote
    def method(self, param:Type, ...):
        ...

Working example:

class Foo:
    @promote
    def method(self, param:Type, ...):
        ...
    # promote method as the final method of the class
    @classmethod
    def promote(cls, obj):
        ... return promoted obj ...

strable(*da, **dkw)

Decorator for functions which may accept a str instead of their core type.

Parameters:

• func: the function to decorate
• open_func: the "open" factory to produce the core type if a string is provided; the default is the builtin "open" function. The returned value should be a context manager. Simpler functions can be decorated with @contextual to turn them into context managers if need be.

The usual (and default) example is a function to process an open file, designed to be handed a file object but which may be called with a filename. If the first argument is a str then that file is opened and the function called with the open file.

Examples:

@strable
def count_lines(f):
  return len(line for line in f)

class Recording:
  "Class representing a video recording."
  ...
@strable(open_func=Recording)
def process_video(r):
  ... do stuff with `r` as a Recording instance ...

Note: use of this decorator requires the cs.pfx module.

uses_cmd_options(*da, **dkw)

A decorator to provide default keyword arguments from the prevailing cs.cmdutils.BaseCommandOptions if available, otherwise from option_defaults.

This exists to provide plumbing free access to options set up by a command line invocation using cs.cmdutils.BaseCommand.

If no option_defaults are provided, a single options keyword argument is provided which is the prevailing BaseCommand.Options instance.

The decorator accepts two optional "private" keyword arguments commencing with underscores:

• _strict: default False; if true then an option_defaults will only be applied if the argument is missing from the function arguments, otherwise it will be applied if the argument is missing or None
• _options_param_name: default 'options'; this is the name of the single options keyword argument which will be supplied if there are no option_defaults

Examples:

@uses_cmd_options(doit=True, quiet=False)
def func(x, *, doit, quiet, **kw):
    if not quiet:
        print("something", x, kw)
    if doit:
       ... do the thing ...
    ... etc ...

@uses_cmd_options()
def func(x, *, options, **kw):
    if not options.quiet:
        print("something", x, kw)
    if options.doit:
       ... do the thing ...
    ... etc ...

uses_doit(func)

func is not supplied, collect the arguments supplied and return a decorator which takes the subsequent callable and returns deco(func, *da, **kw).

uses_force(func)

func is not supplied, collect the arguments supplied and return a decorator which takes the subsequent callable and returns deco(func, *da, **kw).

uses_quiet(func)

func is not supplied, collect the arguments supplied and return a decorator which takes the subsequent callable and returns deco(func, *da, **kw).

uses_verbose(func)

func is not supplied, collect the arguments supplied and return a decorator which takes the subsequent callable and returns deco(func, *da, **kw).

verbose(func)

A decorator to run func if the ambient verbosity is >=1.

verbosity(*da, **dkw)

A decorator to run func if the ambient verbosity is >= verbosity_level.

vv(func)

A decorator to run func if the ambient verbosity is >=2.

with_(func, obj)

A decorator to run func inside a with obj:.

Example:

T = Thread(target=with_(some_context, target_func))

Classes

class Promotable

A mixin class which supports the @promote decorator by providing a default .promote(cls,obj) class method.

Promotable classes also present the .promote(obj) method via __class_getitem__, allowing MyClass[promotable_value].

Promotable.__class_getitem__(index)

Indexing a Promotable class uses the .promote() method to obtain an instance.

Promotable.promote(obj, **from_t_kw)

Promote obj to an instance of cls or raise TypeError. This method supports the @promote decorator.

This base method will call the from_typename(obj,**from_t_kw) class factory method if present, where typename is obj.__class__.__name__.

Subclasses may override this method to promote other types, typically:

@classmethod
def promote(cls, obj):
    if isinstance(obj, cls):
        return obj
    ... various specific type promotions
    ... not done via a from_typename factory method
    # fall back to Promotable.promote
    return super().promote(obj)

An typical from_typename` factory method:

class Foo(Promotable):

    def __init__(self, dbkey, dbrow):
        self.key = dbkey
        self.row_data = row

    @classmethod
    def from_str(cls, s : str):
        """Accept a database key string, return a `Foo` instance."""
        row = db_lookup(s)
        return cls(s, row)

This supports using @promote on functions with Foo instances:

@promote
def do_it(foo : Foo):
    ... work with foo ...

but calling it as:

do_it("foo_key_value")