Skip to main content

A syntactically-aware search and replace tool.

Project description

Refex - refactoring expressions

Refex is a syntactically aware search-and-replace tool for Python, which allows you to specify code searches and rewrites using templates, or a more complex Clang-LibASTMatcher-like matcher interface.

Examples

Automatic parenthesis insertion: Refex will automatically insert parentheses to preserve the intended code structure:

$ echo "a = b.foo() * c" > test.py
$ refex --mode=py.expr '$x.foo()' --sub='$x.foo() + 1' -i test.py
...
$ cat test.py
a = (b.foo() + 1) * c

A naive regular expression replacement would have resulted in b.foo() + 1 * c, which is not equivalent, and is unrelated to the intended replacement.

Paired parentheses: Refex is aware of the full syntax tree, and will always match parentheses correctly:

$ echo "print(foo(bar(b))" > test.py
$ refex --mode=py.expr 'foo($x)' --sub='foo($x + 1)' -i test.py
...
$ cat test.py
a = print(foo(bar(b) + 1))

Here, a naive replacement using regular expressions could have resulted in either print(foo(bar(b)) + 1) or print(foo(bar(b) + 1)), depending on whether $x was matched greedily or non-greedily.

Combining replacements: you can pass multiple search/replace pairs to Refex which combine to do more complex rewrites. For example:

# Rewrites "self.assertTrue(x == False)" to "self.assertFalse(x)", even though
# that was not explicitly called out.
refex --mode=py.expr -i --iterate \
  --match='self.assertTrue($x == $y)'  --sub='self.assertEqual($x, $y)' \
  --match='self.assertEqual($x, False)' --sub='self.assertFalse($x)' \
  -R dir/

TODO: also describe --mode=py.

Getting started

Installation

Refex can be run via pipx for one-off use with control over the Python version:

$ pipx run refex --help

For longer-term use, or for use of Refex as a library, it is also pip-installable:

$ python3 -m venv my_env
$ source my_env/bin/activate
$ pip install refex
$ refex --help

Use

The template syntax is almost exactly what it looks like, so the examples at the top of this page, in combination with the --help output, are intended to be enough to get started.

For more details on the template syntax, see Python Patterns and Templates. For details on how to use refex in your own code as a library, see Using Refex as a Library.

Current status

Stable:

The APIs documented at https://refex.readthedocs.io/ are expected to remain mostly the same, except for trivial renames and moves.

These command-line interfaces are expected to remain roughly the same, without backwards-incompatible changes:

  • --mode=py.expr
  • --mode=fix
  • --mode=re

Unstable

  • All undocumented APIs (especially the API for creating a new matcher).
  • --mode=py.stmt is missing many safety and convenience features.
  • --mode=py, the matcher interface, will eventually need some fairly large restructuring to make it O(n), although simple uses should be unaffected.

(Also, all the stable parts are unstable too. This isn't a promise, just an expectation/statement of intent.)

Contributing

See the contribution guide

See Also

  • asttokens: the token-preserving AST library that Refex is built on top of.
  • Pasta: a code rewriting tool using AST mutation instead of string templates.
  • Semgrep: cross-language AST search using a similar approach.
  • lib2to3: the standard library's code rewriting tool based on the concrete syntax tree.

Disclaimer

You may have noticed Google copyright notices. This is not an officially supported Google product.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

refex-0.1.1.tar.gz (225.7 kB view details)

Uploaded Source

Built Distribution

refex-0.1.1-py3-none-any.whl (163.4 kB view details)

Uploaded Python 3

File details

Details for the file refex-0.1.1.tar.gz.

File metadata

  • Download URL: refex-0.1.1.tar.gz
  • Upload date:
  • Size: 225.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.1.6 CPython/3.9.2 Linux/5.10.28-1rodete1-amd64

File hashes

Hashes for refex-0.1.1.tar.gz
Algorithm Hash digest
SHA256 68487e0fd708988ae3a1a586a63c322d8fc819cff75ad136c1919330c7833ffa
MD5 30960a407a2be4bf363e830b0a99db48
BLAKE2b-256 f81073cc03c82eb9a6424f531b27d1450cc4c37c271f7bf24812b546bc1555eb

See more details on using hashes here.

File details

Details for the file refex-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: refex-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 163.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.1.6 CPython/3.9.2 Linux/5.10.28-1rodete1-amd64

File hashes

Hashes for refex-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 10d935915054efcf5142b2fbe7bba8a635ae370d57dbcdfc5aa7dc111843588e
MD5 e1c52f88a152d048e049e54258df4949
BLAKE2b-256 461c0e5e2a53a9d5cdf09da06cb62b8fd1c9792dda6f0cec3aa6ee246da1bb27

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page