arcangelo 2.1.0

Source code mass automatic upgrade

Overview

arcangelo is an automatic editor for mass transforming python source code. arcangelo is used to perform basic text transformation based on user rules. While in some ways similar to an editor which permits scripted edits (such as ed or sed), arcangelo works by making editor command from rule files. Rule files are simple yaml files, read by arcangelo which try to apply all rules to all lines of files processed.

In this way migration workflow is very simple, accurate, precise and fast.

Usage

usage: arcangelo.py [-h] [-a] [-B] [-b TO_VERSION] [-C RULE_GROUPS] [-c]
                    [-F FROM_VERSION] [-f] [-G GIT_ORGID]
                    [--git-merge-conflict left|right] [--ignore-pragma] [-i]
                    [-j PYTHON] [-l] [-n] [-o OUTPUT] [-P PACKAGE_NAME]
                    [-R RULES] [-S] [--test-res-msg TEST_RES_MSG] [-v] [-V]
                    [-w] [--add-rule-group ADD_RULE_GROUP]
                    [path ...]

Beautiful source file

positional arguments:
  path

options:
  -h, --help            show this help message and exit
  -a, --lint-anyway     set to True when migrate software
  -B, --debug           add comment with applied rule: do not use in
                        production
  -b TO_VERSION, --to-version TO_VERSION
  -C RULE_GROUPS, --rule-groups RULE_GROUPS
                        Rule groups (comma separated) to parse (use + for
                        adding, - for removing) use switch -l to see default
                        groups list
  -c, --copyright-check
  -F FROM_VERSION, --from-version FROM_VERSION
  -f, --force           Parse file even containing '# flake8: noqa' or '#
                        pylint: skip-file'
  -G GIT_ORGID, --git-org GIT_ORGID
  --git-merge-conflict left|right
                        Keep left or right side code after git merge conflict
  --ignore-pragma       ignore coding utf-8 declaration
  -i, --in-place
  -j PYTHON, --python PYTHON
                        python version, format #.##, 2-3 use future
  -l, --list-rules      list rule groups (-ll list with rules too, -lll full
                        list)
  -n, --dry-run         do nothing (dry-run)
  -o OUTPUT, --output OUTPUT
  -P PACKAGE_NAME, --package-name PACKAGE_NAME
  -R RULES, --rules RULES
                        Rules (comma separated) to parse (use - for removing)
                        use switch -ll to see default rules list
  -S, --string-normalization
                        force double quote enclosing strings
  --test-res-msg TEST_RES_MSG
  -v, --verbose
  -V, --version         show program's version number and exit
  -w, --no-parse-with-formatter
                        do nor execute black or prettier on modified files
  --add-rule-group ADD_RULE_GROUP
                        Add rule group form file, default is .arcangelo.yml

© 2021-2025 by SHS-AV s.r.l.

arcangelo is based on rules files located in config directory where arcangelo is running. Configuration files are yaml formatted.

Every rule is list of following format:

PYEREX, (ACTION, PARAMETERS), …

where

• PYEREX is (python expression + enhanced regular expression) for applying the rule

• ACTION is the action to apply on current item (if PYEREX is matched)

• PARAMETERS are the values supplying to action

The list/tuple (ACTION, PARAMETERS) can be repeated more than once under PYEREX

PYEREX is (python expression + enhanced regular expression) is a set of 3 distinct expressions, which are:

1. Python expression (in order to apply eregex): enclosed by double braces

2. Status eregex match (in order to apply eregex): enclosed by parens

3. Applicable eregex to match item

ACTION is applied if (python expression AND status eregex AND applicable eregex); the undeclared python expression or undeclared status eregx returns always true.

eregex is a regular expression (python re) that may be negative if it starts with ! (exclamation mark)

Examples:

Pos	Example	Note	Action
1	REGEX	REGEX is a python re	item is processed if it matches REGEX
2	!REGEX	REGEX is a python re	item is processes if it does not match REGEX
3	\!REGEX	REGEX is a python re beginning with ! (exclamation point)	like case 1
4	!(RE)REGEX	RE and REGEX are two python re	if item does not match (by search) the RE, apply rule 1
5	{{EXPR}}EREGEX	EXPR is double expression	EREGEX is processed if pythonic EXPR is true

• !(import xyz)import -> Rules is applied if matches the statemente “import” but not “import zyz”

• {{self.to_major_version>10}}import something -> If target Odoo version is >10.0 matches statement “import something”, otherwise ignore rule

• {{self.from_major_version<=10}}import something -> If original Odoo version is <=10.0 matches statement “import something”, otherwise ignore rule

• {{self.python_version==3.10}}open -> If python version is 3.10, matches statemente import, otherwise ignore rule

• {{self.py23==3}}open -> If python major version is 3, matches statemente import, otherwise ignore rule

ACTION is the action will be executed when EREGEX is True or when EREGEX fails if action begins with “/” (slash).

ACTION values:

• s: substitute REGEX REPLACE_TEXT

• d: delete line; stop immediately rule processing and re-read the line

• i: insert line before current line

• a: append line after current line

• $: execute FUNCTION

• +: set trigger TRIGGER_NAME (from 1st group of matching regex)

• -: reset trigger TRIGGER_NAME

• =: execute python code

Python test and replacing macros.

Above you can find some simple example of python expression. The following table contains the list of values can used in python expression or in text replacement for substitute action. For example, the value classname can be used in following python expression:

{\{self.classname=="MyClass"}}

while in replacement text the form is:

's' super() super(%(classname)s)

Value list:

Name	Description
backport_multi	Processing a backported version (multiple version path)
classname	Name of current class
dedent	Dedent statement level
final	Processing final version when multiple version path
first_line	True if current line is the 1st of source (see header too)
from_major_version	Major version of project by -F switch
header	Current line is in the file header (comments and empty lines)
imported	Imported packages list
indent	Space indentation of current line
migration_multi	Processing a migrate version with multiple version path
mime	Current file mime
open_stmt	# of open parens; if > 0, current line is a continuation line
python_future	True if source is python 2 and 3 with future
stage	Parsing stage: pre,header,import,class_body,function_body,comment
stmt_indent	Space indentation of current statement
to_major_version	Major version of project by -b switch
transition_stage	Prior parsing stage
try_indent	try statement indentation: if >=0 current line is inside try/except block
py23	Value 2 if python2 else 3 (int)

Action substitute: “s REGEX REPLACE_TEXT”

• The 1.st item is the EREGEX to search for replace (negate is not applied)

• The 2.nd item is the text to replace which can contain macros like %(classname)s

Action delete: “d”

• Delete current line

• Break rules analyzing

Action insert: “i text”

• Insert text before current line

Action append: “a text”

• Append text after current line

Action execute: “$ FUNCTION”

• Function must return requires break and line offset

• If function requires break, no other rules will be processed

• The value 0 for offset means read next line, the value -1 re-read the current line, +1 skip next line, and so on

Function example:

def FUNCTION(self, nro):
    do_break = False
    offset = 0
    if self.lines[nro] == "<odoo>":
        do_break = True
        offset = 1
    return do_break, offset

Rules examples:

Follow rule replace “@api.one” with “# @api.one” and adds comment line:

no_api_mix:
  match: '^ *@api\.(one|returns|cr|model_cr|model_cr_context|v8|noguess)'
  do:
    - action: 's'
      args:
      - '@api\.(one|returns|cr|model_cr|model_cr_context|v8|noguess)'
      - '# @api.\1'
    - action: 'a'
      args:
      - '# TODO> Update code to multi or add self.ensure_one()'

Getting started

Prerequisites

Zeroincombenze(R) tools requires:

• Linux Centos 7/8 or Debian 9/10/11 or Ubuntu 16/18/20/22/24

• python 2.7+, some tools require python 3.7+, best python 3.9+

• bash 5.0+

Installation

Current version via Git

cd $HOME
[[ ! -d ./tools ]] && git clone https://github.com/zeroincombenze/tools.git
cd ./tools
./install_tools.sh -pUT
source $HOME/devel/activate_tools

Upgrade

Current version via Git

cd ./tools
./install_tools.sh -pUT
source $HOME/devel/activate_tools

ChangeLog History

2.1.0 (2025-06-15)

• [IMP] Split from wok_code

• [IMP] Before migration warns on different base name

• [FIX] If target direcotry does not exist, will be create

2.0.22 (2025-05-31)

• [FIX] arcangelo: sometimes wrong format .rst files

2.0.18 (2024-07-10)

• [IMP] Python 3.6 deprecated

2.0.15 (2024-02-17)

• [IMP] arcangelo improvements: new tests odoo from 8.0 to 17.0

• [IMP] arcangelo improvements: test odoo from 8.0 to 17.0

• [IMP] arcangelo switch -lll

• [IMP] arcangelo: rules reorganization

• [IMP] arcangelo: trigger management and new param ctx

• [IMP] arcangelo: new switch -R to select rules to apply

2.0.14 (2024-02-07)

• [FIX] Quality rating formula

• [IMP] arcangelo improvements

2.0.13 (2023-11-27)

• [IMP] arcangelo: new python version assignment from odoo version

2.0.12 (2023-08-29)

• [IMP] arcangelo: new rules

• [IMP] arcangelo: new git conflict selection

• [IMP] arcangelo: merge gen_readme.py formatting

• [IMP] arcangelo: new switch –string-normalization

2.0.10 (2023-07-10)

• [IMP] arcangelo: new switch –string-normalization

2.0.9 (2023-06-26)

• [IMP] arcangelo: refactoring to run inside pre-commit

2.0.2 (2022-10-20)

• [IMP] Clearing code

2.0.1 (2022-10-12)

• [IMP] minor improvements

2.0.1 (2022-10-12)

• [IMP] stable version

2.0.0 (2022-08-10)

• [REF] Refactoring

Credits

Copyright

SHS-AV s.r.l. <https://www.shs-av.com/>

Authors

• SHS-AV s.r.l.

Contributors

• Antonio M. Vigliotti