Edit multiple files using Python text processing modules
formerly known as Python Mass Editor
Implements a python mass editor to process text files using Python code. The modification(s) is (are) shown on stdout as a diff output. One can then modify the target file(s) in place with the -w/–write option. This is very similar to 2to3 tool that ships with Python 3.
|WARNING: A word of caution about the usage of eval()|
This tool is useful as far as it goes but it does rely on the python eval() function and does not check the code being executed. It is a major security risk and one should not use this tool in a production environment.
See Ned Batchelder’s article for a thorough discussion of the dangers linked to eval() and ways to circumvent them. Note that None of the counter-measure suggested in the article are implemented at this time.
You probably will need to know the basics of the Python re module (regular expressions).
usage: massedit.py [-h] [-V] [-w] [-v] [-e EXPRESSIONS] [-f FUNCTIONS] [-x EXECUTABLES] [-s START_DIRS] [-m MAX_DEPTH] [-o FILE] [-g FILE] [--encoding ENCODING] [--newline NEWLINE] [file pattern [file pattern ...]] Python mass editor positional arguments: file pattern shell-like file name patterns to process or - to read from stdin. optional arguments: -h, --help show this help message and exit -V, --version show program's version number and exit -w, --write modify target file(s) in place. Shows diff otherwise. -v, --verbose increases log verbosity (can be specified multiple times) -e EXPRESSIONS, --expression EXPRESSIONS Python expressions applied to target files. Use the line variable to reference the current line. -f FUNCTIONS, --function FUNCTIONS Python function to apply to target file. Takes file content as input and yield lines. Specify function as [module]:?<function name>. -x EXECUTABLES, --executable EXECUTABLES Python executable to apply to target file. -s START_DIRS, --start START_DIRS Directory(ies) from which to look for targets. -m MAX_DEPTH, --max-depth-level MAX_DEPTH Maximum depth when walking subdirectories. -o FILE, --output FILE redirect output to a file -g FILE, --generate FILE generate stub file suitable for -f option --encoding ENCODING Encoding of input and output files --newline NEWLINE Newline character for output files Examples: # Simple string substitution (-e). Will show a diff. No changes applied. massedit.py -e "re.sub('failIf', 'assertFalse', line)" *.py # File level modifications (-f). Overwrites the files in place (-w). massedit.py -w -f fixer:fixit *.py # Will change all test*.py in subdirectories of tests. massedit.py -e "re.sub('failIf', 'assertFalse', line)" -s tests test*.py # Will transform virtual methods (almost) to MOCK_METHOD suitable for gmock (see https://github.com/google/googletest). massedit.py -e "re.sub(r'\s*virtual\s+([\w:<>,\s&*]+)\s+(\w+)(\([^\)]*\))\s*((\w+)*)(=\s*0)?;', 'MOCK_METHOD(\g<1>, \g<2>, \g<3>, (\g<4>, override));', line)" gmock_test.cpp
If massedit is installed as a package (from pypi for instance), one can interact with it as a command line tool:
python -m massedit -e "re.sub('assertEquals', 'assertEqual', line)" test.py
Or as a library (command line option above to be passed as kewyord arguments):
>>> import massedit >>> filenames = ['massedit.py'] >>> massedit.edit_files(filenames, ["re.sub('Jerome', 'J.', line)"])
Lastly, there is a convenient massedit.bat wrapper for Windows included in the distribution.
Download massedit.py from http://github.com/elmotec/massedit or :
pip install massedit
Poor man source-to-source manipulation
I find myself using massedit mostly for source to source modification of large code bases like this:
First create a fixer.py python module with the function that will process your source code. For instance, to add a header:
def add_header(lines, file_name): yield '// This is my header' # will be the first line of the file. for line in lines: yield line
Adds the location of fixer.py to your $PYTHONPATH, then simply call massedit.py like this:
massedit.py -f fixer:add_header *.h
You can add the -s . option to process all the .h files reccursively.
- Add support for 3rd party tool (e.g. autopep8) to process the files.
- Add support for a file of expressions as an argument to allow multiple modification at once.
- Find a satisfactory way (ie. easy to use) to handle multiline regex as the current version works on a line by line basis.
- I have a hard time practicing more than a few dialects of regular expressions.
- I need something portable to Windows without being bothered by eol.
- I believe Python is the ideal tool to build something more powerful than simple regex based substitutions.
I have been using runsed and checksed (from Unix Power Tools) for years and did not find a good substitute under Windows until I came across Graham Fawcett python recipe 437932 on ActiveState. It inspired me to write the massedit.
The core was fleshed up a little, and here we are. If you find it useful and enhance it please, do not forget to submit patches. Thanks!
If you are more interested in awk-like tool, you probably will find pyp a better alternative.
Licensed under the term of MIT License. See attached file LICENSE.txt.
- 0.68.6 (2019-12-02)
- Added support for Python 3.8, stdin input via - argument. Documented regex to turn base classes into googlemock MOCK_METHOD.
- 0.68.5 (2019-04-13)
- Added –newline option to force newline output. Thanks @ALFNeT!
- 0.68.4 (2017-10-24)
- Fixed bug that would cause changes to be missed when the -w option is ommited. Thanks @tgoodlet!
- 0.68.3 (2017-09-20)
- Added –generate option to quickly generate a fixer.py template file to be modified to be used with -f fixer.fixit option. Added official support for Python 3.6
- 0.68.1 (2016-06-04)
- Fixed encoding issues when processing non-ascii files. Added –encoding option to force the value of the encoding if need be. Listed support for Python 3.5
- 0.67.1 (2015-06-28)
- Documentation fixes.
- 0.67 (2015-06-23)
- Added file_name argument to processing functions. Fixed incorrect closing of sys.stdout/stderr. Improved diagnostic when the processing function does not take 2 arguments. Swapped -v and -V option to be consistent with Python. Pylint fixes. Added support for Python 3.4. Dropped support for Python 3.2.
- 0.66 (2013-07-14)
- Fixed lost executable bit with -f option (thanks myint).
- 0.65 (2013-07-12)
- Added -f option to execute code in a separate file/module. Added Travis continuous integration (thanks myint). Fixed python 2.7 support (thanks myint).
- 0.64 (2013-06-01)
- Fixed setup.py so that massedit installs as a script. Fixed eol issues (thanks myint).
- 0.63 (2013-05-27)
- Renamed to massedit. Previous version are still known as Python-Mass-Editor.
- 0.62 (2013-04-11)
- Fixed bug that caused an EditorError to be raised when the result of the expression is an empty string.
- 0.61 (2012-07-06)
- Added massedit.edit_files function to ease usage as library instead of as a command line tool (suggested by Maxim Veksler).
- 0.60 (2012-07-04)
- Treats arguments as patterns rather than files to ease processing of multiple files in multiple subdirectories. Added -s (start directory) and -m (max depth) options.
- 0.52 (2012-06-05)
- Upgraded for python 3. Still compatible with python 2.7.
- 0.51 (2012-05)
- Initial release (Beta).
Release history Release notifications | RSS feed
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size massedit-0.68.6.tar.gz (19.5 kB)||File type Source||Python version None||Upload date||Hashes View|