plib 0.8.9

A namespace package for a number of useful sub-packages and modules.

The PLIB package contains a number of useful sub-packages and modules, all within the plib package namespace in order to minimize clutter in the top-level namespace of your Python installation. Each sub-directory of the plib directory contains a sub-package, except for the test directory, which contains the PLIB test suite. The source distribution also contains an examples directory, which has example programs using PLIB, and a scripts directory, which has a few post-install scripts. Finally, the setup.py script for PLIB uses the setuputils helper module, which helps to automate away much of the boilerplate in Python setup scripts. This module is available as a separate release at http://pypi.python.org/pypi/setuputils.

This version of PLIB is intended to run on the latest Python 2 versions. It has been tested on 2.7, but most of it should run on 2.6 as well; the main known exception is the options module in plib.stdlib, which uses the argparse standard library module that was added in 2.7. If you need to run PLIB on earlier Python versions, the “legacy” version of PLIB is available at http://pypi.python.org/pypi/plib2 as the plib2 package. However, the PLIB API in this and future versions is considerably changed from the “legacy” API, so programs written using plib2 will have to be ported to the new API to use this or future PLIB versions.

The PLIB Sub-Packages

The individual sub-packages and modules contain docstrings with more information about their usage; here they are briefly listed and described.

(Note: This is intended to be the final release of PLIB as a complete package including all its sub-packages. The next PLIB release is intended to be a Beta release of the STDLIB sub-package, followed by the GUI and IO sub-packages. The XML sub-package might or might not be released later; the ``lxml`` package has evolved a lot since PLIB.XML was last tested. The ``plib.gui`` and ``plib.io`` releases will each require ``plib.stdlib``, so ``pip install plib.gui`` or ``pip install plib.io`` should also install ``plib.stdlib``. Splitting the sub-packages this way allows you to only install what you need, and also allows each sub-package to have its own release schedule, independent of the others.)

PLIB.GUI

This sub-package contains a simple GUI application framework with two main features:

• It lets the same high-level code work with a number of different underlying GUI toolkits. Currently supported: Qt (versions 3 and 4), KDE (versions 3 and 4), wxWidgets, and GTK. (The original reason for writing this sub-package was that wxWidgets doesn’t use Qt and I like the Qt/KDE widgets better, but I wanted code that would run cross-platform.)

• It allows you to express the layout of your GUI in terms of Python lists and dicts, enabling a much more declarative and easy to read (and maintain) coding style.

Other than selecting the toolkit (which may not be necessary: the main module of the sub-package can ‘auto-detect’ which toolkit to use–the plib-setup-gui post-install script does most of the work to enable this–so you only need to override if you don’t like the default), you should not have to worry about any toolkit internal details; the goal of this sub-package is to make them all look the same to your code.

Note that the GTK toolkit support in this sub-package is “experimental” and may be removed if it proves to be more trouble than it’s worth. It’s currently included because wxWidgets’ behavior when using GTK as its underlying GUI framework has some quirks that I haven’t been able to work around yet. However, the GTK implementation of a number of widgets (particularly tables and list/tree views) is much less capable than the wxWidgets one, so the Python code for GTK ends up relying much more on ugly hacks.

PLIB.IO

This sub-package contains classes that encapsulate various forms of client/server I/O channels. It is organized into sub-packages itself to make the namespace easier to use. First, the base sub-package contains base classes that implement common basic functionality that is built on by the rest of PLIB.IO.

Most of the remaining sub-packages fall into three main categories, and each sub-package in a given category contains the same basic class names, so they’re easier to remember. The categories are:

• Device Types: socket and serial. Each device type has a BaseClient and BaseServer class; the socket type also has a BaseRequest class. These will usually not need to be used directly; they are used by the I/O mode classes, and are factored out so that each I/O mode sees the same API for a given device type.

• I/O modes: async and blocking (the latter does not just mean synchronous: it includes a forking TCP socket server). Each I/O mode has a client and server class for both device types, and a request class for the socket device type: the class names are SerialClient, SerialServer, SocketClient, SocketServer, and BaseRequestHandler. The async type also has “persistent” classes, which support full-duplex asynchronous communication; these are the PersistentSerial, PersistentSocket, and PersistentRequestHandler classes. Mixin versions of these classes (class names with Mixin at the end) are also provided, for use if alternate data handling is desired (see next bullet), but it is normally not necessary to use these “by hand”–see “automatic mixins” below.

• Data Handling: the I/O mode classes given above include basic data handling, but it is very basic: the only way it can detect that a “message” has been fully received is to detect a closed channel. For some applications this is enough, but often more sophisticated and robust data handling is needed. The data sub-package provides three mixin classes for this purpose, ShutdownReadWrite, TerminatorReadWrite and ReadWrite. The first of these detects the end of a received message by a shutdown of the other end of the data channel, but keeps the channel open to allow further writes (all the other classes default to closing the channel when the other end closes). The other two classes allow the detection of multiple “messages” in the data stream, either by detecting a “terminator” string or by having each message include its length at the beginning. These classes also format outgoing messages the same way.

There is also a mixins sub-package containing classes that are used as mixins by the other sub-packages, and a classes sub-package containing higher-level classes that use the API. Finally, there is a chatgen module which contains a simple class, chat_replies, that yields replies from a remote server as a generator, and a utils module that implements the automatic mixin functionality described below.

Automatic Mixins: To derive your own client or server classes with alternate data handling “by hand”, you would need to use the “mixin” versions of the appropriate I/O mode classes, and splice the data handling class into the middle of the base class list; for example:

from plib.io.async import SerialClientMixin, SerialBase
from plib.io.data import TerminatorReadWrite

class AsyncSerialClientWithTerminator(SerialClientMixin,
    TerminatorReadWrite, SerialBase): pass

This is a bit clumsy, but necessary since the read/write handling has to be before the client/server class in the MRO, but after the base device type, for the cooperative super calls that underlie the functionality to work properly. However, since the pattern is the same in each case, it can be automated, and this has been done in the async and blocking sub-package namespaces, so that instead of doing the above class construction “by hand”, you can just append a suffix to your desired class name, thus:

from plib.io.async import SerialClientWithTerminator

The WithTerminator suffix (or, alternately, WithShutdown or WithReadWrite) will cause the equivalent of the above class definition to occur on the fly, so that the resulting class appears in the plib.io.async namespace (of course the plib.io.blocking namespace has the same capability). Once this has happened the first time, however, the class definition is stored in the appropriate namespace, so additional imports of the same class name (in different modules of your application) will not re-do the “on the fly” construction; they will just retrieve the same class object that was previously constructed.

The above machinery is also made available for use with your own custom read/write handling classes; the async and blocking sub-packages each export a get_readwrite_class function that does the same on-the-fly class definition as above, but with your custom read/write class instead of one of the built-in ones. All you have to do is pass the function the name of your desired I/O class and your custom read/write class object:

from plib.io import async

class CustomReadWrite(object):
    # class definition

MyAsyncSerialClient = async.get_readwrite_class('SerialClient',
    CustomReadWrite)

API Notes: One of the goals of this sub-package is to provide a common, consistent API for all the different types of I/O, so that switching one specific implementation of a certain functionality for another can be done transparently to the rest of your application’s code. Thus, all of the usable classes follow the same basic pattern of mixing in the various pieces of functionality: from left to right in a class’s MRO, one finds the type of endpoint (a client or server mixin class, which may be specialized to the type of I/O), the type of data formatting, if any (a mixin class from the ReadWrite module), and the type of I/O, including device type (socket, serial port, etc.), mode (non-blocking/asynchronous vs. blocking), and basic data handling. Also, each endpoint type has a common API independent of the specific type of I/O and mode; a client can always use the client_communicate method to send data to the server and receive a response; a server can always use the serve_forever method to start itself; and all I/O objects override the same methods to implement application-specific functionality: process_data, to deal with data as it comes in, and query_done, to determine when the I/O channel should be closed. (To see examples of all this in action, look at the test suite in test_io.py and the library module for it, io_testlib.py; the library module can use the same mixin classes to implement test functionality for all of the different mixes of I/O classes in the test suite.)

PLIB.STDLIB

This is a namespace for various functions and classes that extend or emulate the Python standard library. Some, like the cached_property decorator, are implementations of patterns that have been known for some time, but which don’t have a “canonical” version in the stdlib yet; rather than have PLIB depend on some other third-party package, I’ve simply provided my own implementations here. Others, like the abstractcontainer class and its subclasses, are re-implementations of standard Python data structures, written to enable PLIB to make as many things as possible look like those data structures without having to subclass the built-ins (which has some downsides for the use cases I’ve had thus far–see the docstrings for more information).

The following modules or sub-packages are available in the plib.stdlib namespace:

• The builtins module contains some funtions that should be Python builtins, but aren’t. :) Importing the module adds those functions to the built-in namespace; this is mostly useful for interactive shells. The functions can also be imported directly from plib.stdlib.builtins, to make it easier to understand where the functions are coming from in module code.

• The classes sub-package provides some miscellaneous useful classes.

• The classtools module provides some utilities for working with classes and class attributes.

• The cmdline module provides utilities useful for command line programs and interactive shells.

• The coll sub-package provides various collection classes, including abstract collections built on the collections ABCs from the standard library.

• The comm sub-package provides utilities for managing and communicating with child threads and processes.

• The decotools module provides functions and factories for working with decorators.

• The extensions sub-package provides a namespace for functions (and possibly, in the future, other objects) exported from an extension module written using the Python/C API. The general philosophy of PLIB is to do everything possible in pure Python, so the only functions that appear in this sub-package are those which by their very nature cannot be done in pure Python.

• The fdtools module provides utilities for working with file descriptors.

• The imp module provides the import_from_module function, which should be in the standard library importlib module but isn’t. :)

• The ini sub-package implements an abstract ‘INI file’ API that uses ConfigParser on POSIX systems, and the Windows registry on Windows systems. This API allows the configuration file structure to be declared using Python lists and dicts.

• The iters module provides various functions dealing with or returning iterables.

• The localize module provides useful functions for getting locale-specific information.

• The mail module provides a useful shortcut function for sending email from programs.

• The mathlib module provides some additional math functions to supplement those in the standard library.

• The net module provides utilities for getting information about networks.

• The options module provides an easier-to-use overlay for the argparse module which allows you to express your option configuration in the form of Python lists, tuples, and dicts.

• The ostools module provides utilities for working with the operating system.

• The proc module provides a shortcut function for getting the output of a subprocess.

• The sigtools module provides a low-level implementation of the self-pipe trick for signal handling.

• The strings module provides functions and constants for working with strings.

• The systools module exposes some useful variables giving information about the Python runtime system and PLIB itself.

• The timer module provides functions for timing code, with an alternate API to the standard library’s timeit module that is easier to use when timing functions that you already have as objects, instead of source code strings.

• The tztools module provides some useful tzinfo subclasses based on those in the Python docs for the datetime module, and a function to return the local system timezone name.

• The util sub-package provides the ModuleProxy class, which is used by a number of PLIB sub-packages. See the docstrings for the class and the sub-packages using it for more information.

PLIB.XML

This sub-package requires the lxml extension, which uses the very fast libxml2 library but provides a Pythonic API similar to ElementTree. The reason for using lxml instead of ElementTree itself is that lxml has two key additional features:

• Custom element classes: the classes module in this sub-package builds on this feature by using metaclasses to automate DTD generation and validation, but the feature is also great for many XML applications.

• Full and fast XPATH support: this was key in the XML application that first prompted me to write this sub-package. Yes, I know there are plenty of other Python XML packages that do XPATH; the point is to have it plus the standard ElementTree API plus the speed of libxml2.

Installation

To install PLIB, you can simply run:

$ python setup.py install

at a shell prompt from the directory into which you unzipped the source tarball (the same directory that this README file is in). This will install PLIB and then run each of the post-install scripts in the scripts directory.

Example Programs

PLIB comes with example programs that illustrate key features of the package. After installation, these can be found in the $PREFIX/share/plib/examples directory. If you have a POSIX system (Linux or Mac OSX), the plib-setup-examples post-install script will install symlinks to the example programs in the $PREFIX/bin directory.