horast-rereleased 0.4.3

Comment

Full line as well as end-of-line comments are parsed/unparsed correctly when they are outside of multi-line expressions.

Currently, handling of comments within multi-line expressions is implemented only partially.

BlockComment

Not implemented yet.

This node type is intended to store consecutive full-line comments in a single AST node, therefore simplifying handling of large blocks of comments.

Directive

Since Python doesn’t offer a direct way to convey directives in code, comments beginning with any of the following prefixes will not be classified as typical comments, but as directives:

• if

• else

• endif

• def

• undef

• ifdef

• ifndef

However in Python code they still remain as usual comments.

For example, the comments in the following code will all be classified as directives.

#ifdef DEBUG
debugging = True
#else
debugging = False
#fi

However, when executed as Python, debugging will always end up False because directives are preserved as usual comments in Python and therefore they are ignored.

Therefore, the Directive node is not meant to enable preprocessing of Python, at least for now.

Note: the prefix is checked exactly. See the following example:

#if something
# if some other thing

The comment in the first line will become Directive object, while the one on the second like will become Comment object.

Currently, this node is meant to work towards AST compatibility between Python and other languages, to aide code generation from Python AST into code in other languages – one such use case is *transpyle* project.

Pragma

Pragma nodes follow the same general rules as Directive nodes, i.e. they are stored in Python code as comments, but a comment will be classified as a pragma when it’s prefixed with a predefined prefix:

• " pragma:"

Additionally, two subclasses of Pragma are defined in horast, each with its own prefix that builds upon the generic pragma prefix:

• OpenMpPragma class defines prefix " pragma: omp" and stores OpenMP pragmas.

• OpenAccPragma class defines prefix " pragma: acc" and stores OpenACC pragmas.

A code snippet below contains all 3 pragma types.

# pragma: once
use_openmp = True
use_openacc = True
...
a, b = np.ndarray(...)
c = np.zeros(...)
# pragma: acc parallel copyin(a,b) copyout(c)
# pragma: acc loop gang
for y in range(ymax): # type: np.int32
    # pragma: acc loop worker
    for i in range(imax): # type: np.int32
        # pragma: acc loop vector reduction(+: c[y][x])
        for x in range(xmax): # type: np.int32
            c[y, x] += a[y, i] * b[i, x]
# pragma: acc end parallel
...
# pragma: omp parallel do
for i in range(input_data.size):  # type: int
    # here we compute spam spam spam
    heavy_compute(input_data[i])
...

And thus, in the example above:

• all comments starting with " pragma: omp" become OpenMpPragma objects,

• all comments starting with " pragma: acc" become OpenAccPragma objects,

• all other comments starting with " pragma:" become Pragma objects,

• type comments are ignored, and

• all other comments become Comment objects.

Additionally, horast module provides an extensible infrastructure to define custom Pragma subclasses, enabling user to define their own pragmas for experimentation. The provided OpenMP and OpenACC pragma definitions serve as examples of how to use this feature.

Include

Similarly to how pragmas are handled, if a comment begins with " include:" prefix, it will be classified as a special kind of include directive.

Again, this will be preserved as comment in Python code, but it’s useful for enhancing syntactic compatibility between Python and other, especially statically compiled languages.