Skip to main content

Use with to simplify multi-level object dereferences, reminisent of Pascal's with statement

Project description

travisci PyPI Package latest release Supported versions Supported implementations Wheel packaging support Test line coverage Test branch coverage

withref makes Python’s with statement able to simplify complex dereferences. This is what I initially, naively thought the statement was for, similar to the with statement of half-remembered Pascal.

The typical Python use is a more complex guard over the entry and exit points for using an object. See e.g. this effbot article, But withref makes the simpler “just give me the dereferenced object, please” use case work as well.


from __future__ import print_function # Python 2/3 compatibility
from withref import ref
from stuf import stuf

# stuf used for demonstration purposes only. Convenient way to
# make dict entries accessible as dot-deferenceable properties.

a = stuf({ 'b': { 'c': { 'c1': 1 }, 'd': 44.1 } })

with ref(a.b.c) as c:
    c.c1 = 99

a_ideal = stuf({'b': {'c': {'c1': 99}, 'd': 44.1}})
assert a == a_ideal

It works with array-style references too, of course:

with ref(a['b']['c']) as cc:
    c['c1'] = 99

assert a == a_ideal

But beware OVER-dereferencing! While this works correctly:

with ref(a.b.c.c1) as c1:
    assert c1 == 99

This does NOT:

with ref(a.b.c.c1) as c1:
    c1 = 12345
    assert c1 == 12345

assert a.b.c.c1 == 99   # with heavy heart
                        # because we just ostensibly set it to 12345

Unlike Pascal, ref is not built into the language proper. And unlike Perl, Python is less eager to provide lvalues for every mention of a variable or value. As a result, ref can provide the value of the full-deferenced c1, but not the assignable lvalue.

Like Pascal’s with, ref removes N-1 layers of the enclosing structure for values, but only N-2 for assignment. Still, in complex multi-layer structures, this can be a nice simplification:

with ref(app.config.server.wsgi) as wsgi:
    wsgi.logger = some_logger
    wsgi.debug_level = 4
    wsgi.port = 8080

arguably beats:

app.config.server.wsgi.logger = some_logger
app.config.server.wsgi.debug_level = 4
app.config.server.wsgi.port = 8080

for simplicity and clarity.

I see a lot of configuration code that constantly repeats the same long multi-level dereferences. That style is repetitive (anti-DRY) and tends to left-align code blocks, both of which impede program comprehension. Using the with statement is a neat way to simplify and at the same time add a little extra visual structure.

And while assignment (“lvalue production”) isn’t always possible, there still are some interesting tricks possible with simple value production:

with ref("this is a string"[0:4]) as t:


While yadda.yadda.yadda referencing is all too common, one can also use a more proximate variable assignment:

wsgi = app.config.server.wsgi
wsgi.logger = some_logger
wsgi.debug_level = 4
wsgi.port = 8080

This lacks the more indented structure of the withref approach, but is still much preferable to what you often find in the field.

See Also

The withhacks module, which includes many other fun hacks (multi-line lambdas, new looping structures, etc.)–but also requires the byteplay module which actively introspects & munges Python bytecode (“Danger, Will Robinson! Danger!”), and has not been updated to work beyond Python 2.6. So it’s out of currency with all but the very tail-end of modern Python.

Possible Future Extension

As withhacks shows, using introspection we could determine the lvalue of the calling object even in the edge-case where it is a leaf-node of its enclosing structure. This would not require any bytecode changes, and should be compatible with modern versions of Python (e.g. 2.7] and 3.x).

Whether that trick can be done simply, portably, rock-solid reliably, and transparently enough to satisfy those who code the modules that tend to most need this kind of dereferencing simplification–i.e. complex modules often used in production settings, into which they are understandably loathe to introduce any possible sources of error or any performance impedance–that is the key open question.


  • Automated multi-version testing managed with pytest, pytest-cov, coverage and tox. Packaging linting with pyroma.

    Successfully packaged for, and tested against, all late-model versions of Python: 2.6, 2.7, 3.3, 3.4, 3.5, 3.6, as well as recent PyPy and PyPy3 builds.

  • The author, Jonathan Eunice or @jeunice on Twitter welcomes your comments and suggestions.


To install or upgrade to the latest version:

pip install -U withref

You may need to prefix these with sudo to authorize installation. In environments without super-user privileges, you may want to use pip’s --user option, to install only for a single user, rather than system-wide. Depending on your local system configuration, you may also need version-specific installers such as pip2 and pip3.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution (38.1 kB view hashes)

Uploaded Source

Built Distribution

withref-0.3.3-py2.py3-none-any.whl (8.3 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page