Skip to main content

Container for flexible class, instance, and function call options

Project description

A module that helps encapsulate option and configuration data using a multi-layer stacking (a.k.a. nested context) model.

Classes are expected to define default option values. When instances are created, they can be instantiated with “override” values. For any option that the instances doesn’t override, the class default “shines through” and remains in effect. Similarly, individual method calls can set transient values that apply just for the duration of that call. If the call doesn’t set a value, the instance value applies. If the instance didn’t set a value, the class default applies. Python’s with statement can be used to tweak options for essentially arbitrary duration.

This layered or stacked approach is particularly helpful for highly functional classes that aim for “reasonable” or “intelligent” defaults and behaviors, that allow users to override those defaults at any time, and that aim for a simple, unobtrusive API. It can also be used to provide flexible option handling for functions.

This option-handling pattern is based on delegation rather than inheritance. It’s described in this StackOverflow.com discussion of “configuration sprawl”.

Unfortunately, it’s a bit hard to demonstrate the virtues of this approach with simple code. Python already supports flexible function arguments, including variable number of arguments (*args) and optional keyword arguments (**kwargs). Combined with object inheritance, base Python features already cover a large number of use cases and requirements. But when you have a large number of configuration and instance variables, and when you might want to temporarily override either class or instance settings, things get dicey. This messy, complicated space is where options truly begins to shine.

Usage

from options import Options, attrs

class Shape(object):

    options = Options(
        name   = None,
        color  = 'white',
        height = 10,
        width  = 10,
    )

    def __init__(self, **kwargs):
        self.options = Shape.options.push(kwargs)

    def draw(self, **kwargs):
        opts = self.options.push(kwargs)
        print attrs(opts)

one = Shape(name='one')
one.draw()
one.draw(color='red')
one.draw(color='green', width=22)

yielding:

color='white', width=10, name='one', height=10
color='red', width=10, name='one', height=10
color='green', width=22, name='one', height=10

So far we could do this with instance variables and standard arguments. It might look a bit like this:

class ClassicShape(object):

    def __init__(self, name=None, color='white', height=10, width=10):
        self.name   = name
        self.color  = color
        self.height = height
        self.width  = width

but when we got to the draw method, things would be quite a bit messier.:

def draw(self, **kwargs):
    name   = kwargs.get('name',   self.name)
    color  = kwargs.get('color',  self.color)
    height = kwargs.get('height', self.height)
    width  = kwargs.get('width',  self.width)
    print "color='{}', width={}, name='{}', height={}".format(color, width, name, height)

One problem here is that we broke apart the values provided to __init__() into separate instance variables, now we need to re-assemble them into something unified. And we need to explicitly choose between the **kwargs and the instance variables. It gets repetitive, and is not pretty. Another classic alternative, using native keyword arguments, is no better:

def draw2(self, name=None, color=None, height=None, width=None):
    name   = name   or self.name
    color  = color  or self.color
    height = height or self.height
    width  = width  or self.width
    print "color='{}', width={}, name='{}', height={}".format(color, width, name, height)

If we add just a few more instance variables, we have the Mr. Creosote of class design on our hands. Not good. Things get worse if we want to set default values for all shapes in the class. We have to rework every method that uses values, the __init__ method, et cetera. We’ve entered “just one more wafer-thin mint…” territory.

But with options, it’s easy:

Shape.options.set(color='blue')
one.draw()
one.draw(height=100)
one.draw(height=44, color='yellow')

yields:

color='blue', width=10, name='one', height=10
color='blue', width=10, name='one', height=100
color='yellow', width=10, name='one', height=44

In one line, we reset the default for all Shape objects.

The more options and settings a class has, the more unwieldy the class and instance variable approach becomes, and the more desirable the delegation alternative. Inheritance is a great software pattern for many kinds of data and program structures, but it’s a bad pattern for complex option and configuration handling. For richly featured classes, the delegation pattern options proves simpler. Supporting even a large number of options requires almost no additional code and imposes no additional complexity or failure modes. By consolidating options into one place, and by allowing neat, attribute-style access, everything is kept tidy. We can add new options or methods with confidence:

def is_tall(self, **kwargs):
    opts = self.options.push(kwargs)
    return opts.height > 100

Under the covers, options uses a variation on the ChainMap data structure (a multi-layer dictionary) to provide its option stacking. Every option set is stacked on top of previously set option sets, with lower-level values shining through if they’re not set at higher levels. This stacking or overlay model resembles how local and global variables are managed in many programming languages.

Magic Parameters

Python’s *args variable-number of arguments and **kwargs keyword arguments are sometimes called “magic” arguments. options takes this up a notch, allowing setters much like Python’s property function or @property decorator. This allows arguments to be interpreted on the fly. This is useful, for instance, to provide relative rather than just absolute values. As an example, say that we added this code after Shape.options was defined:

options.magic(
    height = lambda v, cur: cur.height + int(v) if isinstance(v, str) else v,
    width  = lambda v, cur: cur.width  + int(v) if isinstance(v, str) else v
)

Now, in addition to absolute height and width parameters which are provided by specifying int (integer/numeric) values, your module auto-magically supports relative parameters for height and width.:

one.draw(width='+200')

yields:

color='blue', width=210, name='one', height=10

This can be as fancy as you like, defining an entire domain-specific expression language. But even small functions can give you a great bump in expressive power. For example, add this and you get full relative arithmetic capability (+, -, *, and /):

def relmath(value, currently):
    if isinstance(value, str):
        if value.startswith('*'):
            return currently * int(value[1:])
        elif value.startswith('/'):
            return currently / int(value[1:])
        else:
            return currently + int(value)
    else:
        return value

...

options.magic(
    height = lambda v, cur: relmath(v, cur.height),
    width  = lambda v, cur: relmath(v, cur.width)
)

Then:

one.draw(width='*4', height='/2')

yields:

color='blue', width=40, name='one', height=5

Magically interpreted parameters are the sort of thing that one doesn’t need very often or for every parameter–but when they’re useful, they’re enormously useful and highly leveraged, leading to much simpler, much higher function APIs. We call them ‘magical’ here because of the “auto-magical” interpretation, but they are really just analogs of Python object properties. The magic function is basically a “setter” for a dictionary element.

Design Considerations

In general, classes will define a set of methods that are “outwards facing”–methods called by external code when consuming the class’s functionality. Those methods should generally expose their options through **kwargs, creating a local variable (say opts) that represents the sum of all options in use–the full stack of call, instance, and class options, including any present magical interpretations.

Internal class methods–the sort that are not generally called by external code, and that by Python convention are often prefixed by an underscore (_)–these generally do not need **kwargs. They should accept their options as a single variable (say opts again) that the externally-facing methods will provide.

For example, if options didn’t provide the nice formatting function attrs, we might have designed our own:

def _attrs(self, opts):
    nicekeys = [ k for k in opts.keys() if not k.startswith('_') ]
    return ', '.join([ "{}={}".format(k, repr(opts[k])) for k in nicekeys ])

def draw(self, **kwargs):
    opts = self.options.push(kwargs)
    print self._attrs(opts)

draw(), being the outward-facing API, accepts general arguments and manages their stacking (by push``ing ``kwargs onto the instance options). When the internal _attrs() method is called, it is handed a pre-digested opts package of options.

A nice side-effect of making this distinction: Whenever you see a method with **kwargs, you know it’s outward-facing. When you see a method with just opts, you know it’s internal.

Objects defined with options make excellent “callables.” Define the __call__ method, and you have a very nice analog of function calls.

options has broad utility, but it’s not for every class or module. It best suits high-level front-end APIs that multiplex lots of potential functionality, and wish/need to do it in a clean/simple way. Classes for which the set of instance variables is small, or functions/methods for which the set of known/possible parameters is limited–these work just fine with classic Python calling conventions. For those, options is overkill. “Horses for courses.”

Setting and Unsetting

Using options, objects often become “entry points” that represent both a set of capabilities and a set of configurations for how that functionality will be used. As a result, you may want to be able to set the object’s values directly, without referencing their underlying options. It’s convenient to add a set() method, then use it, as follows:

def set(self, **kwargs):
    self.options.set(**kwargs)

one.set(width='*10', color='orange')
one.draw()

yields:

color='orange', width=100, name='one', height=10

one.set() is now the equivalent of one.options.set(). Or continue using the options attribute explicitly, if you prefer that.

Values can also be unset.:

from options import Unset

one.set(color=Unset)
one.draw()

yields:

color='blue', width=100, name='one', height=10

Because 'blue' was the color to which Shape had be most recently set. With the color of the instance unset, the color of the class shines through.

NOTA BENE while options are ideally accessed with an attribute notion, the preferred of setting options is through method calls: set() if accessing directly, or push() if stacking values as part of a method call. These perform the interpretation and unsetting magic; straight assignment does not. In the future, options may provide an equivalent __setattr__() method to allow assignment–but not yet.

Leftovers

options expects you to define all feasible and legitimate options at the class level, and to give them reasonable defaults.

None of the initial settings ever have magic applied. Much of the expected interpretation “magic” will be relative settings, and relative settings require a baseline value. The top level is expected and demanded to provide a reasonable baseline.

Any options set “further down” such as when an instance is created or a method called should set keys that were already-defined at the class level.

However, there are cases where “extra” **kwargs values may be provided and make sense. Your object might be a very high level entry point, for example, representing very large buckets of functionality, with many options. Some of those options are relevant to the current instance, while others are intended as pass-throughs for lower-level modules/objects. This may seem a doubly rarefied case–and it is, relatively speaking. But it does happen, and when you need multi-level processing, it’s really, really super amazingly handy to have it.

options supports this in its core push() method by taking the values that are known to be part of the class’s options, and deleting those from kwargs. Any values left over in the kwargs dict are either errors, or intended for other recipients.

As yet, there is no automatic check for leftovers.

The Magic APIs

The callables (usually functions, lambda expressions, static methods, or methods) called to preform magical interpretation can be called with 1, 2, or 3 parameters. options inquires as to how many parameters the callable accepts. If it accepts only 1, it will be the value passed in. Cleanups like “convert to upper case” can be done, but no relative interpretation. If it accepts 2 arguments, it will be called with the value and the current option mapping, in that order. (NB this order reverses the way you may think logical. Caution advised.) If the callable requires 3 parameters, it will be None, value, current mapping. This supports method calls, though has the defect of not really passing in the current instance.

A decorator form, magical() is also supported. It must be given the name of the key exactly:

@options.magical('name')
def capitalize_name(self, v, cur):
    return ' '.join(w.capitalize() for w in v.split())

The net is that you can provide just about any kind of callable. But the meta-programming of the magic interpretation API could use a little work.

Subclassing

Subclass options may differ from superclass options. Usually they will share many options, but some may be added, and others removed. To modify the set of available options, the subclass defines its options with the add() method to the superclass options. This creates a layered effect, just like push() for an instance. The difference is, push() does not allow new options (keys) to be defined; add() does. It is also possible to assign the special null object Prohibited, which will disallow instances of the subclass from setting those values.:

options = Superclass.options.add(
    func   = None,
    prefix = Prohibited,  # was available in superclass, but not here
    suffix = Prohibited,  # ditto
)

Because some of the “additions” can be prohibitions (i.e. removing particular options from being set or used), this is “adding to” the superclass’s options in the sense of “adding a layer onto” rather than strict “adding options.”

An alternative is to copy (or restate) the superclass’s options. That suits cases where the subclass is highly independent, and where changes to the superclass’s options should not effect the subclass’s options. With add(), they remain linked in the same way as instances and classes are.

Transients

Some options do not make sense as permanent values–they are needed only as transient settings in the context of individual calls. The special null value Transient can be assigned as an option value to signal this.

Flat Arguments

Sometimes it’s more elegant to provide some arguments as flat, sequential values rather than by keyword. In this case, use the addflat() method:

def __init__(self, *args, **kwargs):
    self.options = Quoter.options.push(kwargs)
    self.options.addflat(args, ['prefix', 'suffix'])

to consume optional prefix and suffix flat arguments. This makes the following equivalent:

q1 = Quoter('[', ']')
q2 = Quoter(prefix='[', suffix=']')

An explicit addflat() method is provided not as much for Zen of Python reasons (“Explicit is better than implicit.”), but because flat arguments are commonly combined with abbreviation/shorthand conventions, which may require some logic to implement. For example, if only a prefix is given as a flat argument, you may want to use the same value to implicitly set the suffix. To this end, addflat returns the set of keys that it consumed:

if args:
    used = self.options.addflat(args, ['prefix', 'suffix'])
    if 'suffix' not in used:
        self.options.suffix = self.options.prefix

Notes

  • This is a work in progress. The underlying techniques have been successfully used in multiple projects, but it remains in an evolving state as a standalone module. The API may change over time. Swim at your own risk.

  • Open question: Could “magic” parameter processing be improved with a properties-based approach akin to that of basicproperty, propertylib, classproperty, and realproperty.

  • Open question: Should “magic” parameter setters be allow to change multiple options at once? A use case for this: “Abbreviation” options that combine multiple changes into one compact option. These would probably not have stored values themselves. It would require setting the “dependent” option values via side-effect rather than functional return values.

  • The author, Jonathan Eunice or @jeunice on Twitter welcomes your comments and suggestions.

Recent Changes

  • Commenced automated multi-version testing with pytest and tox. Now successfully packaged for, and tested against, Python 2.6, 2.7, 3.2, and 3.3.

  • Options is now packaged for, and tested against, PyPy 1.9 (based on 2.7.2). The underlying stuf module and orderedstuf class is not certified for PyPy, and it exhibits a bug with file objects on PyPy. options works around this bug, and tests fine on PyPy. Still, buyer beware.

  • Versions subsequent to 0.200 require a late-model version of stuf to avoid a problem its earlier iterations had with file objects. Versions after 0.320 depend on stuf for chainstuf, rather than the otherstuf sidecar.

  • Now packaged as a package, not a set of modules. six module now required only for testing.

  • API for push() and addflat() cleaned up to explicitly delink those methods.

Installation

pip install options

To easy_install under a specific Python version (3.3 in this example):

python3.3 -m easy_install options

(You may need to prefix these with “sudo “ to authorize installation.)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

options-0.411.zip (35.9 kB view hashes)

Uploaded Source

options-0.411.tar.gz (18.5 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page