textpy 0.2.4

Reads a python module and statically analyzes it.

textpy

Reads a python module and statically analyzes it. This works well with jupyter extensions in VS Code, and will have better performance when the module files are formatted with PEP-8.

Installation

$ pip install textpy

Requirements

lazyr
typing-extensions
black
colorama
htmlmaster
re_extensions

NOTE: pandas>=1.4.0 is recommended but not necessary.

Quick Start

To demonstrate the usage of this module, we put a file named myfile.py under ./examples/ (you can find it in the repository, or create a new file of your own):

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from typing import Optional


class MyBook:
    """
    A book that records a story.

    Parameters
    ----------
    story : str, optional
        Story to record, by default None.

    """

    def __init__(self, story: Optional[str] = None) -> None:
        if story is None:
            self.content = "This book is empty."
        self.content = story


def print_my_book(book: MyBook) -> None:
    """
    Print a book.

    Parameters
    ----------
    book : MyBook
        A book.

    """
    print(book.content)

Run the following codes to find all the occurrences of some pattern (for example, "MyBook") in myfile.py:

>>> import textpy as tx
>>> myfile = tx.module("./examples/myfile.py") # reads the python module

>>> myfile.findall("MyBook")
examples/myfile.py:7: 'class <MyBook>:'
examples/myfile.py:24: 'def print_my_book(book: <MyBook>) -> None:'
examples/myfile.py:30: '    book : <MyBook>'

If you are using a jupyter notebook, you can run a cell like this:

>>> myfile.findall("MyBook")

source	match
myfile.MyBook:7	class MyBook:
myfile.print_my_book():24	def print_my_book(book: MyBook) -> None:
myfile.print_my_book():30	book : MyBook

In this case, the matched substrings are clickable, linking to where the patterns were found.

Usage

tx.module()

The previous demonstration introduced the core function tx.module(). The return value of tx.module() is a subinstance of the abstract class tx.TextTree, which supports various text manipulation methods:

>>> isinstance(myfile, tx.TextTree)
True

Sometimes, your python module may contain not just one file, but don't worry, since tx.module() provides support for complex file hierarchies. If the path points to a single file, the return type will be PyFile; otherwise, the return type will be PyDir - both of which are subclasses of tx.TextTree.

In conclusion, once you've got a python package, you can simply give the package dirpath to tx.module(), and do things like before:

>>> pkg_dir = "" # type any path here
>>> pattern = "" # type any regex pattern here

>>> res = tx.module(pkg_dir).findall(pattern)

tx.TextTree.findall()

As mentioned before, user can use .findall() to find all non-overlapping matches of some pattern in a python module.

>>> myfile.findall("optional")
examples/myfile.py:13: '    story : str, <optional>'

The object returned by .findall() has a _repr_mimebundle_() method to beautify the representation inside a jupyter notebook. However, you can compulsively disable this feature by setting display_params.use_mimebundle to False:

>>> from textpy import display_params
>>> display_params.use_mimebundle = False

In addition, the .findall() method has some optional parameters to customize the pattern, including whole_word=, case_sensitive=, and regex=.

>>> myfile.findall("mybook", case_sensitive=False, regex=False, whole_word=True)
examples/myfile.py:7: 'class <MyBook>:'
examples/myfile.py:24: 'def print_my_book(book: <MyBook>) -> None:'
examples/myfile.py:30: '    book : <MyBook>'

tx.TextTree.replace()

Use .replace() to find all non-overlapping matches of some pattern, and replace them with another string:

>>> replacer = myfile.replace("book", "magazine")
>>> replacer
examples/myfile.py:9: '    A <book/magazine> that records a story.'
examples/myfile.py:20: '            self.content = "This <book/magazine> is empty."'
examples/myfile.py:24: 'def print_my_<book/magazine>(<book/magazine>: MyBook) -> None:'
examples/myfile.py:26: '    Print a <book/magazine>.'
examples/myfile.py:30: '    <book/magazine> : MyBook'
examples/myfile.py:31: '        A <book/magazine>.'
examples/myfile.py:34: '    print(<book/magazine>.content)'

At this point, the replacement has not actually taken effect yet. Use .confirm() to confirm the changes and write them to the file(s):

>>> replacer.confirm()
{'successful': ['examples/myfile.py'], 'failed': []}

If you want to rollback the changes, run:

>>> replacer.rollback()
{'successful': ['examples/myfile.py'], 'failed': []}

tx.TextTree.delete()

Use .delete() to find all non-overlapping matches of some pattern, and delete them:

>>> deleter = myfile.delete("book")
>>> deleter
examples/myfile.py:9: '    A <book> that records a story.'
examples/myfile.py:20: '            self.content = "This <book> is empty."'
examples/myfile.py:24: 'def print_my_<book>(<book>: MyBook) -> None:'
examples/myfile.py:26: '    Print a <book>.'
examples/myfile.py:30: '    <book> : MyBook'
examples/myfile.py:31: '        A <book>.'
examples/myfile.py:34: '    print(<book>.content)'

>>> deleter.confirm()
{'successful': ['examples/myfile.py'], 'failed': []}

>>> deleter.rollback()
{'successful': ['examples/myfile.py'], 'failed': []}

See Also

Github repository

• https://github.com/Chitaoji/textpy/

PyPI project

• https://pypi.org/project/textpy/

License

This project falls under the BSD 3-Clause License.

History

v0.2.4

• The first argument of tx.module() is optional now, by default "".

v0.2.3

• Compatible with htmlmaster==0.0.4.

v0.2.2

• Compatible with htmlmaster==0.0.2.

v0.2.1

• New module-level functions tx.file() and tx.fromstr().
• Removed the deprecated function tx.textpy().
• Removed utils.re_extensions as a submodule, related utils are now directly import from the package re_extensions. User can use an instance of re_extensions.SmartPattern as the pattern for TextTree.findall(), TextTree.replace(), and TextTree.delete().
• After this version, the required Python version is updated to >=3.12.7. Download and install v0.1.32 if the user is under lower Python version (>=3.8.13).

v0.1.32

• Added dist in DEFAULT_IGNORED_PATHS.

v0.1.31

• Got ANSI escapes to work on Windows.

v0.1.30

• New optional paramter include= for tx.module().
• Renamed tx.PyText to tx.TextTree; the name PyText will be deprecated.

v0.1.29

• Updated PyText.check_format(), which now returns a boolean value instead of None.
• Updated the ignore= parameter for tx.module(), which now accepts a list of path-patterns. Paths matching any of these patterns will be ignored when searching for files.

v0.1.28

• Fixed issue: failed to display special characters in *._repr_mimebundle_().

v0.1.27

• New gloabal parameters: tree_style=, table_style=, use_mimebundle=, and skip_line_numbers= - find them under tx.display_params.
• Defined display_params.defaults() for users to get the default values of the parameters.
• New subclass PyProperty inherited from PyMethod. Class properties will be stored in instances of PyProperty instead of PyMethod in the future.
• Updated the method PyText.jumpto(): it now allows "/" as delimiters (in addition to "."); if a class or callable is defined more than once, jump to the last (previously first) place where it was defined.
• PyText has a _repr_mimebundle_() method now.
• New property PyText.imports.
• Created a utility class HTMLTableMaker in place of Styler; this significantly reduces the running overhead of *._repr_mimebundle_().

v0.1.26

• Updated with the package re_extensions:
  • bugfix for rsplit();
  • new string operation quote_collapse().

v0.1.25

• Updated utils.re_extensions:
  • Important: we've decided to extract utils.re_extensions into an independent package named re_extensions (presently at v0.0.3), so any future updates should be looked up in https://github.com/Chitaoji/re-extensions instead; we will stay in sync with it, however;
  • real_findall() now returns match objects instead of spans and groups;
  • smart_sub() accepts a new optional parameter called count=;
  • SmartPattern supports [] to indicate a Unicode (str) or bytes pattern (like what re.Pattern does);
  • new regex operations smart_split(), smart_findall(), line_findall(), smart_subn(), and smart_fullmatch();
  • created a namespace Smart for all the smart operations;
  • bugfixes for rsplit(), lsplit(), and smart_sub().
• Reduced the running cost of PyText.findall() by taking advantage of the new regex operation line_findall().

v0.1.24

• New methods PyText.is_file() and PyText.is_dir() to find out whether the instance represents a file / directory.
• New method PyText.check_format() for format checking.
• Defined the comparison ordering methods __eq__(), __gt__(), and __ge__() for PyText. They compares two PyText object via their absolute paths.
• Updated utils.re_extensions:
  • new regex operations smart_search(), smart_match(), and smart_sub();
  • new string operation counted_strip();
  • new utility classes SmartPattern and SmartMatch.
  • new utility functions find_right_bracket() and find_left_bracket().

v0.1.23

• New string operation utils.re_extensions.word_wrap().
• Various improvements.

v0.1.22

• The module-level function tx.textpy() is going to be deprecated to avoid conflicts with the package name textpy. Please use tx.module() insead.
• New methods PyText.replace() and PyText.delete().
• New class Replacer as the return type of PyText.replace(), with public methods .confirm(), .rollback(), etc.
• Added a dunder method PyText.__truediv__() as an alternative to PyText.jumpto().
• New subclass PyContent inherited from PyText. A PyContent object stores a part of a file that is not storable by instances of other subclasses.

v0.1.21

• Improved behavior of clickables.

v0.1.20

• Fixed issue: incorrect file links in the output of TextPy.findall();

v0.1.19

• Various improvements.

v0.1.18

• Updated LICENSE.

v0.1.17

• Refactored README.md.

v0.1.16

• Lazily imported pandas to reduce the time cost for importing.

v0.1.12

• New optional parameters for TextPy.findall() :
  • whole_word= : whether to match whole words only;
  • case_sensitive= : specifies case sensitivity.

v0.1.10

• New optional parameter encoding= for textpy().

v0.1.9

• Removed unnecessary dependencies.

v0.1.8

• Bugfix under Windows system.

v0.1.5

• Provided compatibility with pandas versions lower than 1.4.0.
• Updated textpy() :
  • Path object is now acceptable as the positional argument;
  • new optional parameter home= for specifying the home path.
• More flexible presentation of output from TextPy.findall().

v0.1.4

• Fixed a display issue of README.md on PyPI.

v0.1.3

• Initial release.