Ceygen 0.3

Cython helper for linear algebra with typed memoryviews built atop the Eigen C++ library

About

Ceygen is a binary Python extension module for linear algebra with Cython typed memoryviews. Ceygen is built atop the Eigen C++ library. Ceygen is not a Cython wrapper or an interface to Eigen!

The name Ceygen is a rather poor wordplay on Cython + Eigen; it has nothing to do with software piracy. Ceygen is currently distributed under GNU GPL v2+ license. The authors of Ceygen are however open to other licensing suggestions. (Do you want to use Ceygen in e.g. a BSD-licensed project? Ask!)

Cython is being developed by Matěj Laitl with support from the Institute of Information Theory and Automation, Academy of Sciences of the Czech Republic. Feel free to send me a mail to matej at laitl dot cz.

Features

Ceygen…

• is fast - Ceygen’s primary raison d’être is to provide overhead-free algebraic operations for Cython projects that work with typed memoryviews (especially small-sized). For every function there is a code-path where no Python function is called, no memory is allocated on heap and no data is copied. Eigen itself performs rather well, too.

• is documented - see Documentation or hop directly to on-line docs.

• supports various data types - Ceygen uses Cython fused types (a.k.a. wannabe templates) along with Eigen’s template nature to support various data types without duplicating code. While just a few types are pre-defined (float, double, …), adding a new type is a matter of adding 3 lines and rebuilding Ceygen.

• is extensively tested - Ceygen’s test suite validates every its public method, including errors raised on invalid input. Thanks to Travis CI, every push is automatically tested against Python 2.6, 2.7, 3.2 and 3.3.

• is multithreading-friendly - Every Ceygen function doesn’t acquire the GIL unless it needs to create a Python object (always avoidable); all functions are declared nogil so that you can call them in prange blocks without losing parallelism.

• provides descriptive error messages - Care is taken to propagate all errors properly (down from Eigen) so that you are not stuck debugging your program. Ceygen functions don’t crash on invalid input but rather raise reasonable errors.

• works well with NumPy, but doesn’t depend on it. You don’t need NumPy to build or run Ceygen, but thanks to Cython, Cython memoryviews and NumPy arrays are fully interchangeable without copying the data (where it is possible). The test suite currently makes use of NumPy because of our laziness. :-)

On the other hand, Ceygen…

• depends on Eigen build-time. Ceygen expects Eigen 3 headers to be installed under /usr/lib/eigen3 when it is being built. Installing Eigen is a matter of unpacking it, because it is a pure template library defined solely in the headers. Ceygen doesn’t reference Eigen at all at runtime because all code is complited in.

• still provides a very little subset of Eigen functionality. We add new functions only as we need them in another projects, but we believe that the hard part is the infrastructure - implementing a new function should be rather straightforward (with decent Cython and C++ knowledge). We’re very open to pull requests! (do include unit tests in them)

• needs recent Cython (currently at least 0.19.1) to compile. If this is a problem, you can distribute .cpp files or final Python extension module instead.

• doesn’t bring Eigen’s elegance to Cython - if you think of lazy evaluation and advanced expessions, stop dreaming. Ceygen will make your code faster, not nicer. Array expessions will help here.

A simple example to compute matrix product within a big matrix may look like

>>> cdef double[:, :] big = np.array([[1.,  2.,   2.,  0.,   0.,  0.],
>>>                                   [3.,  4.,   0., -2.,   0.,  0.]])
>>> ceygen.core.dot_mm(big[:, 0:2], big[:, 2:4], big[:, 4:6])
[[ 2. -4.]
 [ 6. -8.]]
>>> big
[[ 1.  2.   2.  0.   2. -4.]
 [ 3.  4.   0. -2.   6. -8.]],

where the dot_mm call above doesn’t copy any data, allocates no memory on heap, doesn’t need the GIL and uses vectorization (SSE, AltiVec…) to get the best out of your processor.

Obtaining

Ceygen development happens in its github repository, git clone [e-mail]:strohel/Ceygen.git -ing is the preferred way to get it as you’ll have the latest & greatest version (which shouldn’t break thanks to continuous integration). Released versions are available from Ceygen’s PyPI page.

Building

Ceygen uses standard Distutils to build, test and install itself, simply run:

• python setup.py build to build Ceygen

• python setup.py test to test it (inside build directory)

• python setup.py install to install it

• python setup.py clean to clean generated object, .cpp and .html files (perhaps to force recompilation)

Commands can be combined, automatically call dependent commands and can take options, the recommended combo to safely install Ceygen is therefore python setup.py -v test install.

Building Options

You can set various build options as it is usual with distutils, see python setup.py --help. Notable is the build_ext command and its –include-dirs (standard) and following additional options (whose are Ceygen extensions):

--include-dirs

defaults to /usr/include/eigen3 and must be specified if you’ve installed Eigen 3 to a non-standard directory,

--cflags

defaults to -O2 -march=native -fopenmp. Please note that it is important to enable optimizations and generation of appropriate MMX/SSE/altivec-enabled code as the actual computation code from Eigen is built along with the boilerplate Ceygen code,

--ldflags

additional flags to pass to linker, defaults to -fopenmp. Use standard –libraries for specifying extra libraries to link against,

--annotate

pass –annotate to Cython to produce annotated HTML files during compiling. Only useful during Ceygen development.

You may want to remove -fopenmp from cflags and ldflags if you are already parallelising above Ceygen. The resulting command could look like python setup.py -v build_ext --include-dirs=/usr/local/include/eigen3 --cflags="-O3 -march=core2" --ldflags= test. The same could be achieved by putting the options to a setup.cfg file:

[build_ext]
include_dirs = /usr/local/include/eigen3
cflags = -O3 -march=core2
ldflags =

Documentation

Ceygen documentation is maintained in reStructuredText format under doc/ directory and can be exported into a variety of formats using Sphinx (version at least 1.0 needed). Just type make in that directory to see a list of supported formats and for example make html to build HTML pages with the documentation.

See ChangeLog.rst file for changes between versions or view it online.

On-line documentation is available at http://strohel.github.com/Ceygen-doc/

Bugs

Please report any bugs you find and suggestions you may have to Ceygen’s github Issue Tracker.