pyTooling 8.0.3

pyTooling is a powerful collection of arbitrary useful classes, decorators, meta-classes and exceptions.

pyTooling

pyTooling is a powerful collection of arbitrary useful abstract data models, missing classes, decorators, a new performance boosting meta-class and enhanced exceptions. It also provides lots of helper functions e.g. to ease the handling of package descriptions or to unify multiple existing APIs into a single API.

It's useful - if not even essential - for any Python-base project independent if it's a library, framework, CLI tool or just a "script".

In addition, pyTooling provides a collection of CI job templates for GitHub Actions. This drastically simplifies GHA-based CI pipelines for Python projects.

Package Details

Attributes

The [pyTooling.Attributes] module offers the base implementation of .NET-like attributes realized with Python decorators. The annotated and declarative data is stored as instances of Attribute classes in an additional field per class, method or function.

The annotation syntax (decorator syntax) allows users to attach any structured data to classes, methods or functions. In many cases, a user will derive a custom attribute from Attribute and override the init method, so user-defined parameters can be accepted when the attribute is constructed.

Later, classes, methods or functions can be searched for by querying the attribute class for attribute instance usage locations (see example to the right). Another option for class and method attributes is declaring a classes using pyTooling’s ExtendedType meta-class. Here the class itself offers helper methods for discovering annotated methods.

A SimpleAttribute class is offered accepting any positional and keyword parameters. In a more advanced use case, users are encouraged to derive their own attribute class hierarchy from Attribute.

Use Cases

In general all classes, methods and functions can be annotated with additional meta-data. It depends on the application, framework or library to decide if annotations should be applied imperatively as regular code or declaratively as attributes via Python decorators.

With this in mind, the following use-cases and ideas can be derived:

• Describe a command line argument parser (like ArgParse) in a declarative form. |br| See pyTooling.Attributes.ArgParse Package and Examples
• Mark nested classes, so later when the outer class gets instantiated, these nested classes are indexed or automatically registered.
  See CLIAbstraction → [CLIABS/CLIArgument]
• Mark methods in a class as test cases and classes as test suites, so test cases and suites are not identified based on a magic method name.
  Investigation ongoing / planned feature.

Using SimpleAttribute

from pyTooling.Attributes import SimpleAttribute

@SimpleAttribute(kind="testsuite")
class MyClass:
  @SimpleAttribute(kind="testcase", id=1, description="Test and operator")
  def test_and(self):
    ...

  @SimpleAttribute(kind="testcase", id=2, description="Test xor operator")
  def test_xor(self):
    ...

CLI Abstraction

[pyTooling.CLIAbstraction] offers an abstraction layer for command line programs, so they can be used easily in Python. There is no need for manually assembling parameter lists or considering the order of parameters. All parameters like -v or --value=42 are described as [CommandLineArgument] instances on a [Program] class. Each argument class like [ShortFlag] or [PathArgument] knows about the correct formatting pattern, character escaping, and if needed about necessary type conversions. A program instance can be converted to an argument list suitable for [subprocess.Popen].

While a user-defined command line program abstraction derived from [Program] only takes care of maintaining and assembling parameter lists, a more advanced base-class, called [Executable], is offered with embedded [subprocess.Popen] behavior.

Design Goals

• Offer access to CLI programs as Python classes.
• Abstract CLI arguments (a.k.a. parameter, option, flag, ...) as members on such a Python class.
• Abstract differences in operating systems like argument pattern (POSIX: -h vs. Windows: /h), path delimiter signs (POSIX: / vs. Windows: \) or executable names.
• Derive program variants from existing programs.
• Assemble parameters as list for handover to [subprocess.Popen] with proper escaping and quoting.
• Launch a program with :class:[subprocess.Popen] and hide the complexity of Popen.
• Get a generator object for line-by-line output reading to enable postprocessing of outputs.

Common Helper Functions

This is a set of useful helper functions:

• getsizeof calculates the "real" size of a data structure.
• isnestedclass checks if a class is nested inside another class.
• firstKey, firstValue, firstPair get the firstItem key/value/item from an ordered dictionary.
• mergedicts merges multiple dictionaries into a new dictionary.
• zipdicts iterate multiple dictionaries simultaneously.

Common Classes

• Call-by-reference parameters: Python doesn't provide call-by-reference parameters for simple types.
  This behavior can be emulated with classes provided by the pyTooling.CallByRef module.
• Unified license names: Setuptools, PyPI, and others have a varying understanding of license names.
  The pyTooling.Licensing module provides unified license names as well as license name mappings or translations.
• Unified platform and environment description: Python has many ways in figuring out the current platform using APIs from sys, platform, os, …. Unfortunately, none of the provided standard APIs offers a comprehensive answer. pyTooling provides a CurrentPlatform singleton summarizing multiple platform APIs into a single class instance.
• Representations of version numbers: While Python itself has a good versioning schema, there are no classes provided to abstract version numbers. pyTooling provides such representations following semantic versioning (SemVer) and calendar versioning (CalVer) schemes. It's provided by the pyTooling.Versioning module.

Configuration

Various file formats suitable for configuration information share the same features supporting: key-value pairs (dictionaries), sequences (lists), and simple types like string, integer and float. pyTooling provides an abstract configuration file data model supporting these features. Moreover, concrete configuration file format reader implementations are provided as well.

• JSON configuration reader for the JSON file format.
• TOML configuration reader → To be implemented.
• YAML configuration reader for the YAML file format.

Data Structures

pyTooling also provides fast and powerful data structures offering object-oriented APIs:

• Graph data structure
  → A directed graph implementation using a Vertex and an Edge class.
• Path data structure
  → To be documented.
• Finite State Machine data structure
  → A data model for state machines using a State and a Transition class.
• Tree data structure
  → A fast and simple implementation using a single Node class.

Decorators

• Abstract Methods
  • Methods marked with abstractmethod are abstract and need to be overwritten in a derived class.
    An abstract method might be called from the overwriting method.
  • Methods marked with mustoverride are abstract and need to be overridden in a derived class.
    It's not allowed to call a mustoverride method.
• Documentation
  • Copy the doc-string from given base-class via InheritDocString.
• Visibility
  • Register the given function or class as publicly accessible in a module via export.
• Documentation
  • @InheritDocString
    → Copy the doc-string from given base-class.
• Visibility
  • @export
    → Register the given function or class as publicly accessible in a module.

Exceptions

• EnvironmentException
  ... is raised when an expected environment variable is missing.
• PlatformNotSupportedException
  ... is raise if the platform is not supported.
• NotConfiguredException
  ... is raise if the requested setting is not configured.

Meta-Classes

pyTooling provides an enhanced meta-class called ExtendedType. This meta-classes allows to implement abstract methods, singletons, slotted types and combinations thereof.

class MyClass(metaclass=ExtendedType): A class definition using that meta-class can implement abstract methods using decorators @abstractmethod or @mustoverride.

class MyClass(metaclass=ExtendedType, singleton=True): A class defined with enabled singleton behavior allows only a single instance of that class to exist. If another instance is going to be created, a previously cached instance of that class will be returned.

class MyClass(metaclass=ExtendedType, slots=True): A class defined with enabled slots behavior stores instance fields in slots. The meta-class, translates all type-annotated fields in a class definition into slots. Slots allow a more efficient field storage and access compared to dynamically stored and accessed fields hosted by __dict__. This improves the memory footprint as well as the field access performance of all class instances. This behavior is automatically inherited to all derived classes.

class MyClass(ObjectWithSlots): A class definition deriving from ObjectWithSlots will bring the slotted type behavior to that class and all derived classes.

Packaging

A set of helper functions to describe a Python package for setuptools.

• Helper Functions:
  • loadReadmeFile
    Load a README.md file from disk and provide the content as long description for setuptools.
  • loadRequirementsFile
    Load a requirements.txt file from disk and provide the content for setuptools.
  • extractVersionInformation
    Extract version information from Python source files and provide the data to setuptools.
• Package Descriptions
  • DescribePythonPackage
    tbd
  • DescribePythonPackageHostedOnGitHub
    tbd

Terminal

A set of helpers to implement a text user interface (TUI) in a terminal.

Features

• Colored command line outputs based on colorama.
• Message classification in fatal, error, warning, normal, quiet, ...
• Get information like terminal dimensions from underlying terminal window.

Simple Terminal Application

This is a minimal terminal application example which inherits from LineTerminal.

from pyTooling.TerminalUI import TerminalApplication

class Application(TerminalApplication):
  def __init__(self) -> None:
    super().__init__()

  def run(self):
    self.WriteNormal("This is a simple application.")
    self.WriteWarning("This is a warning message.")
    self.WriteError("This is an error message.")

# entry point
if __name__ == "__main__":
  Application.CheckPythonVersion((3, 6, 0))
  app = Application()
  app.run()
  app.Exit()

Timer

tbd

Examples

@export Decorator

from pyTooling.Decorators import export

@export
class MyClass:
  pass

CallByRefIntParam

from pyTooling.CallByRef import CallByRefIntParam

# define a call-by-reference parameter for integer values
myInt = CallByRefIntParam(3)

# a function using a call-by-reference parameter
def func(param: CallByRefIntParam):
  param <<= param * 4

# call the function and pass the wrapper object
func(myInt)

print(myInt.Value)

Contributors

• Patrick Lehmann (Maintainer)
• Sven Köhler
• Unai Martinez-Corral
• and more...

License

This Python package (source code) licensed under Apache License 2.0.
The accompanying documentation is licensed under Creative Commons - Attribution 4.0 (CC-BY 4.0).

---

SPDX-License-Identifier: Apache-2.0