convtools 0.3.0

convtools is a python library to declaratively define fast conversions from python objects to python objects, including processing collections and doing complex aggregations.

convtools is a python library to declaratively define fast conversions from python objects to python objects, including processing collections and doing complex aggregations.

Description

The speed of convtools comes from the approach of generating code & compiling conversion functions, which don’t have any generic code like superfluous loops, ifs, etc.

So you can follow the DRY principle by storing and reusing the code on the python expression level, but at the same time be able to run the gen_converter and get the compiled code which doesn’t care about being DRY and is generated to be highly specialized for the specific need.

Thanks to pipes & labels it’s possible to define multiple pipelines of data processing, including branching and merging of them.

Conversions are not limited to simple data transformations, there are GroupBy & Aggregate conversions with many useful reducers:

• from common Sum, Max

• and less widely supported First/Last, Array/ArrayDistinct

• to DictSum-like ones (for nested aggregation) and MaxRow/MinRow (for finding an object with max/min value and further processing)

Every conversion:

• contains the information of how to transform an input

• can be piped into another conversion (same as wrapping)

• can be labeled to be reused further in the conversions chain

• has a method gen_converter returning a function compiled at runtime

• despite being compiled at runtime, is debuggable with pdb due to linecache populating.

Installation:

pip install convtools

Documentation

convtools on Read the Docs