camel-snake-pep8 0.2.3

Use Python-Rope to convert names in a Python project to conform with PEP-8.

camel-snake-pep8

A refactoring tool to help convert camel case to snake case and vice versa in a Python program, in conformity with the PEP-8 style guide. It uses/abuses Python-Rope to find and perform the changes. The program interactively displays proposed changes and the code diffs that would result from the change. It queries the user as to whether or not to accept the changes.

The program does not do all the changes for full PEP-8 naming compliance, but it does most of them. It currently does not recognize unpacked assignments to tuples very well, and it does not try to modify any names in the context of import statements (though Rope will indirectly change some names in import statements if they are renamed elsewhere).

Note that a formatting program such as autopep8 (which is pip installable) can be used to automatically fix many syntactical and spacing issues, but those programs do not rename variables. If such a program is also used it should be done as a separate step, and some testing should be done between running the two programs to help isolate any problems which might be introduced.

• Use this software at your own risk. This program has various features to try to avoid introducing errors in renaming, but correctness cannot be guaranteed. Always make a backup copy of any project before running this program on it. The program has been used a few times with good results, but does not currently have formal tests.

  Rope is not perfect, so check your results and look at the warnings issued. Rope can have problems with changing names which are imported from different modules, especially with “import … as”, so it might be a good idea to change all names which appear in such statements by hand.

• Mostly tested on Ubuntu Linux but should also work on Windows.

Installing and using

To install using pip, run:

pip install camel-snake-pep8 --upgrade

A recent version of Rope is recommended for Python 3 compatibility.

The program is a single module, and can also be downloaded and run directly as python camel_snake_pep8.py, provided the dependencies (colorama and rope) are installed.

Usage:

camel-snake-pep8 <projectDir> <moduleToModify> [<moduleToModify> ...]

If the module arguments are omitted the program will gather all the .py files in the <projectDir> directory, and recursively for any subpackages if the directory has an __init__.py file. If all arguments are omitted then the current directory is assumed to be the main project directory.

The program can be used to refactor either Python 2 or Python 3 code. Run the program installed in the same version of Python as the code that is being modified. That is, if the code being refactored is Python 2 code then run the program installed in Python 2, and similarly for Python 3.

Note that Rope currently only has limited support for Python 3 type hinting.

As an example, to change all the Python files in a package with one subpackage go to the package root directory and type:

camel-snake-pep8

or, equivalently:

camel-snake-pep8 . *.py */*.py

If individual modules to modify are listed be sure to also include the paths to any subpackage modules, subsubpackage modules, etc., which are to be modified on the same line. Note that any faulty code in the same directory as the files to be modified may cause Rope to fail, since Rope looks at those also (and may modify them secondary to a change in a selected file).

The program can be stopped at any time with ^C. But note that it is better to make all the changes in one run of the program. That is because the program collects and saves all the names in modules to change, before any changes are made, in order to give warnings about possible name collisions.

If you want to quickly convert a (copy of a) full project at once to see what the results would be, two command-line options are provided. The option --yes-to-all runs the program as if the user had entered “y” to all queries. The option --yes-no-default runs the program as if the user just hit return, giving the default action (which is to accept changes without warnings and reject others). The latter is safer, but in either case running without reviewing the changes may result in some changes that are unintended.

How it works

This program goes through each file character by character, keeping the character offset value. This offset is passed to Python-Rope to detect variables to possibly rename. The program queries the user about proposed changes and makes any user-approved changes. Python-Rope is also used to do the renaming.

The names and offsets from a module file are all re-calculated after each change, since offsets can change with each modification. The running time is nevertheless not bad for interactive use. Variable names for rejected name changes — which keep the original variable name — are temporarily renamed to have a magic string appended to them. This is so the program knows the name has been reviewed and should be retained. This magic string is then globally removed from all the files after all the possible changes are processed. If the program halts abnormally (such that the finally of a try is not executed) some of those magic strings may still be present.

Warnings and theory

The program tries to make the refactoring as safe as possible, since bugs introduced by bad renaming can be difficult to find. One of the main dangers with renaming operations is name collisions.

One type of name collision occurs because Rope will happily rename a variable to a name that is already in use in the same scope. For example, a function parameter could be renamed to collide with a preexisting local variable inside the function. Here is an example:

def f(camelArg):
    camelArg = 555
    camel_arg = 444
    return camelArg

If the change of the parameter camelArg to camel_arg is accepted (despite the warning that will be issued) the new function will return 444 instead of the previous value 555. The camel-snake-pep8 program will issue a warning since the new name previously existed in the module before any changes were made (i.e, before any changes by the current run of the program).

Another type of name collision is when the renaming itself causes two distinct names like myVar and myVAR to map to a common new name my_var. In this case, a warning is given if a name change that was accepted (on the current run of the program) already mapped a different name to that same new name.

Warnings are issued for possible situations which may lead to a collision – or may not, since scoping is not taken into account in the analysis. The default query reply, such as when the user just hits “enter” each time, is to accept the change when no warning is given and reject the change when a warning is given. Many of the changes with warnings will actually be safe, but before accepting one the displayed diffs for the change (and possibly the files themselves) should be carefully inspected to be sure. As an alternative, a different name entirely can be tried by hitting c in response to the query.

After all the changes are made the program does an analysis looking for potential problems, and warnings are issued for any that are found. No scoping is taken into account so many of these warnings are probably false alarms. To be cautious, though, the warnings should still be checked to see what is causing them.

Another problem comes when Rope changes the name of a variable assigned in a module, but then fails to also change an import statement from another module which imports that variable from the first module. Similarly, Rope cannot resolve some attribute assignments. Both of these kinds of problems will generate warnings after all the changes have been made.

To summarize: all names per module are saved before any changes, and all names per module are saved after all the changes. The name mappings are all saved. A warning is given on mapping a name into a name that pre-existed in a module. A warning is also given for a mapping that collides with a previous mapping (i.e., is not one-to-one). After all the changes, the places where preimages of accepted-change mappings still exist are warned about. Similarly, places where the images of rejected-change mappings still exist are warned about.

Rough “proof” of reasonable safety for changes without warnings, assuming that Python-Rope does the name replacements correctly (which it doesn’t always do, e.g., class attributes it cannot resolve).

1. The camel case strings that this program would change to snake case strings without issuing a warning (and vice versa) are disjoint sets of names.

2. If no occurrences of the new, proposed name exist in any file where changes are made then no warning will be given and all the instances of the old name will be converted to the new one. No name collisions can occur because the new name did not exist in any of those files in the first place. Any variables which end up with the same name already had the same name in the first place.

Of course since Python is dynamic and has introspection there will always be cases where the rename substitutions fail (such as modifying the globals dict). Rope is also not perfect, and fails to make some changes which it should make for semantic equivalence. Most of these latter errors will at least cause a warning to be generated after all the changes have been applied.

License

Copyright (c) 2017 by Allen Barker. MIT license, see the file LICENSE for more details.