pandalone 0.1.0

process data-trees with reconfigurable-paths

pandalone: process data-trees with reconfigurable-paths

Release:

0.1.0

Documentation:

https://pandalone.readthedocs.org/

Source:

https://github.com/pandalone/pandalone

PyPI repo:

https://pypi.python.org/pypi/pandalone

Keywords:

calculation, data, dependencies, engineering, excel, library, numpy, pandas, processing, python, resolution, scientific, simulink, tree, utility

Copyright:

2015 European Commission (JRC-IET)

License:

EUPL 1.1+

pandalone is an open source Python 2/3 library for building component-functions to process hierarchical-data using reconfigurable-paths.

Our goal is to facilitate the composition of engineering-models from loosely-coupled components. Initially envisioned as an indirection-framework around pandas coupled with a dependency-resolver, every such model should auto-adapt and process only values available, and allow remapping of the paths accessing them, to run on renamed/relocated value-trees without component-code modifications.

It is written for python-3.4 but tested under both python-2.7 and python-3.3+, for Windows and Linux.

Note

The project, as of May-2015, is considered at an alpha-stage, without any released version in pypi yet.

Table of Contents

• pandalone: process data-trees with reconfigurable-paths

  • Introduction

    • Overview

    • Quick-start

Introduction

Overview

At the most fundamental level, an “execution” or a “run” of a processing can be thought like that

      .--------------.     _____________        .-------------.
     ;  DataTree    ;    |             |      ;   DataTree   ;
    ;--------------; ==> |  <cfunc_1>  | ==> ;--------------;
   ; /some/data   ;      |  <cfunc_2>  |    ; /some/data   ;
  ;  /some/other ;       |     ...     |   ;  /some/other ;
 ;   /foo/bar   ;        |_____________|  ;   /foo/bar   ;
'--------------'                         '--------------.

• The data-tree might come from json, hdf5, excel-workbooks, or plain dictionaries and lists. Its values are strings and numbers, numpy-lists, pandas or xray-datasets, etc.

• The component-functions must abide to the following simple signature:

  cfunc_do_something(pandelone, datatree)

  and must not return any value, just read and write into the data-tree.

• Here is a simple component-function:

  def cfunc_standardize(pandelone, datatree):
      pin, pon = pandelone.paths(),
      df = datatree.get(pin.A)
      df[pon.A.B_std] = df[pin.A.B] / df[pin.A.B].std()

• Notice the use of the reconfigurable-paths marked specifically as input or output.

• TODO: continue rough example in tutorial…

Quick-start

Note

The program runs on Python-2.7+ and Python-3.3+ (preferred) and requires numpy/scipy, pandas and win32 libraries along with their native backends to be installed. If you do not have such an environment already installed, please read doc(install) section below for suitable distributions such as WinPython or Anaconda.