plib 0.5.2

A namespace package for a number of useful sub-packages and modules.

The PLIB package contains a number of useful sub-packages and modules, all within the plib package namespace in order to minimize clutter in the top-level namespace of your Python installation. Each sub-directory of the plib directory contains a sub-package, except for the test directory, which contains the PLIB test suite. The source distribution also contains an examples directory, which has example programs using PLIB, and a scripts directory, which has a few post-install scripts. (The SetupHelper directory in the source distribution is not installed; it contains a helper module for PLIB’s setup script. SetupHelper is available as a separate package from PyPI under the name setuphelper.)

The PLIB Sub-Packages

The individual sub-packages and modules contain docstrings with more information about their usage; here they are briefly listed and described.

PLIB.CLASSES

Each of the modules in this sub-package contains a single class with the same name; the ModuleProxy class from the PLIB.UTILS sub-package is used to make all the classes appear in the plib.classes namespace. See the sub-package and ModuleProxy docstrings for more information.

The classes in this sub-package are mostly a miscellaneous group, included for no other reason than that I have found them useful. The only common theme shared by some of them is implementation of common “policy” for I/O clients and servers, on top of the basic mechanisms provided by the I/O channel classes in plib.stdlib.

PLIB.EXTENSIONS

This sub-package provides a namespace for functions (and possibly, in the future, other objects) exported from an extension module written using the Python/C API. The general philosophy of PLIB is to do everything possible in pure Python, so the only functions that appear in this sub-package are those which by their very nature cannot be done in pure Python.

PLIB.GUI

This is the largest sub-package, and contains a simple GUI application framework with two main features:

• It lets the same high-level code work with a number of different underlying GUI toolkits. Currently supported: Qt (versions 3 and 4), KDE, wxWidgets, and GTK. (The original reason for writing this sub-package was that wxWidgets doesn’t use Qt and I like the Qt/KDE widgets better, but I wanted code that would run cross-platform.)

• It allows you to express the layout of your GUI in terms of Python lists and dicts, enabling a much more declarative and easy to read (and maintain) coding style.

Other than selecting the toolkit (which may not be necessary: the main module of the sub-package can ‘auto-detect’ which toolkit to use–the plib-setup-gui post-install script does most of the work to enable this–so you only need to override if you don’t like the default), you should not have to worry about any toolkit internal details; the goal of this sub-package is to make them all look the same to your code.

Note that the GTK toolkit support in this sub-package is “experimental” and may be removed if it proves to be more trouble than it’s worth. It’s currently included because wxWidgets’ behavior when using GTK as its underlying GUI framework has some quirks that I haven’t been able to work around yet. However, the GTK implementation of a number of widgets (particularly tables and list/tree views) is much less capable than the wxWidgets one, so the Python code for GTK ends up relying much more on ugly hacks.

PLIB.INI

This sub-package implements an abstract ‘INI file’ API that uses ConfigParser on POSIX systems, and the Windows registry on Windows systems. As with the PLIB.GUI sub-package, the goal is to hide the internal details of the configuration storage from your code, so all you have to do is define your configuration structure, again using native Python data types (lists and dicts).

PLIB.STDLIB

This is a namespace for various functions and classes that extend or emulate the Python standard library. Some, like the cached_property decorator, are implementations of patterns that have been known for some time, but which don’t have a “canonical” version in the stdlib yet; rather than have PLIB depend on some other third-party package, I’ve simply provided my own implementations here. Others, like the abstractcontainer class and its subclasses, are re-implementations of standard Python data structures, written to enable PLIB to make as many things as possible look like those data structures without having to subclass the built-ins (which has some downsides for the use cases I’ve had thus far–see the docstrings for more information).

This sub-package also contains modules that extend or provide alternate interfaces to Python standard library modules. These fall into two main categories:

I/O Modules

These modules encapsulate various forms of client/server I/O channels.

• The async module contains base classes that abstract out the core asynchronous I/O functionality of the asyncore module, and make it usable with anything that has a Unix file descriptor, not just sockets. Various other modules (see below) combine this base with classes that encapsulate a basic type of I/O (sockets, serial ports, pipes, etc.) to provide usable clients and servers.

• The AsyncSerial module uses the base functionality of the async and pyserial modules to provide asynchronous I/O using a serial device.

• The AsyncServer module uses the base functionality of the async module to provide a base dispatcher class similar to asyncore.dispatcher, and client/server classes with a similar API to those given in the SocketServer module, but using asynchronous I/O.

• The ClientServer module provides mixin classes for simple blocking I/O clients and servers. These classes can be used with any type of I/O (socket, serial port, etc.) when mixed in with the appropriate classes from other modules in this sub-package.

• The io module provides the BaseData class, which encapsulates basic data handling for all the I/O classes in this sub-package. This class assumes no formatting of the data being read and written; for simple applications this is usually sufficient, but it can be difficult to know for sure when the end of data being read is encountered; the classes in the ReadWrite module can be used to deal with this.

• The pyserial module provides a thin wrapper around the Serial class from the pyserial package, which adds some flexibility to usage to allow working around quirks of various serial devices. Other modules in this sub-package use this module to provide basic serial device I/O.

• The ReadWrite module provides two alternate options for handling and formatting data read and written over an I/O channel, the object being to make it easier to determine when all of the data for a particular “message” sent over the channel has been read, so that the process_data method can be called to deal with it. The TerminatorReadWrite class looks for a specific terminator string in the data stream, and calls process_data when it is found. The ReadWrite class prepends the length of each “message” to the message itself, and unpacks the length at the other end so it knows precisely how many bytes are in each message.

• The SerialIO module builds on the ClientServer and pyserial modules to provide basic blocking I/O using a serial device.

• The SigSocketServer module provides an alternate implementation of the forking TCP server from the Python standard library, which looks for SIGCHLD to know when to reap child processes for request handlers that have completed. This module also provides a BaseRequestHandler class using classes from the socketio and ClientServer modules (this request handler can use blocking I/O since it will be run from a forking server in a separate child process).

• The SocketClient module provides the ClientSocketMixin and SocketClient classes, which combine classes from the socketio and ClientServer modules to provide basic blocking socket client functionality. (Because this client is blocking, it is mostly useful for forked sub-processes that can block without disturbing the main application. If a non-blocking socket client is needed, use the ClientSocketMixin or ClientCommunicator class from the AsyncServer module instead.)

• The socketio module encapsulates the basic functionality needed for I/O using sockets, both blocking and non-blocking.

One of the goals of the I/O modules is to provide a common, consistent API for all the different types of I/O, so that switching one specific implementation of a certain functionality for another can be done transparently to the rest of your application’s code. Thus, all of the usable classes follow the same basic pattern of mixing in the various pieces of functionality: from left to right in a class’s MRO, one finds the type of endpoint (a client or server mixin class, which may be specialized to the type of I/O), the type of data formatting, if any (a mixin class from the ReadWrite module), and the type of I/O, including device type (socket, serial port, etc.), mode (non-blocking/asynchronous vs. blocking), and basic data handling. Also, each endpoint type has a common API independent of the specific type of I/O and mode; a client can always use the client_communicate method to send data to the server and receive a response; a server can always use the serve_forever method to start itself; and all I/O objects override the same methods to implement application-specific functionality: process_data, to deal with data as it comes in, and query_done, to determine when the I/O channel should be closed. (To see examples of all this in action, look at the test suite in test_stdlib_io.py and the library module for it, stdlib_io_testlib.py; the library module can use the same mixin classes to implement test functionality for all of the different mixes of I/O classes in the test suite.)

Customized Classes and Overlays

These modules provide customized versions of standard library classes, or overlays that make standard library functionality easier to use.

• The coll module provides two slightly customized collection classes, fifo and stack, with a common API.

• The options module provides an easier-to-use overlay for the optparse module which allows you to express your option configuration in the form of Python lists, tuples, and dicts, and also adds some minimal argument checking functionality.

PLIB.UTILS

This sub-package contains some miscellaneous useful functions and modules, and also the ModuleProxy class referred to above.

PLIB.XML

This sub-package requires the lxml extension, which uses the very fast libxml2 library but provides a Pythonic API similar to ElementTree. The reason for using lxml instead of ElementTree itself is that lxml has two key additional features:

• Custom element classes: the classes module in this sub-package builds on this feature by using metaclasses to automate DTD generation and validation, but the feature is also great for many XML applications.

• Full and fast XPATH support: this was key in the XML application that first prompted me to write this sub-package. Yes, I know there are plenty of other Python XML packages that do XPATH; the point is to have it plus the standard ElementTree API plus the speed of libxml2.

Installation

To install PLIB, you can simply run:

$ python setup.py install

at a shell prompt from the directory into which you unzipped the source tarball (the same directory that this README file is in). This will install PLIB and then run each of the post-install scripts in the scripts directory.

Example Programs

PLIB comes with example programs that illustrate key features of the package. After installation, these can be found in the $PREFIX/share/plib/examples directory. If you have a POSIX system (Linux or Mac OSX), the plib-setup-examples post-install script will install symlinks to the example programs in the $PREFIX/bin directory.