Skip to main content

Bonobo

Project description

🐵 bonobo
=========

Data-processing. By monkeys. For humans.

Bonobo is a data-processing library for python 3.5+ that emphasis writing
simple, atomic, plain old python functions and chaining them using a basic
acyclic graph. The nodes will need a bit of plumbery to be runnable in
different means (iteratively, in threads, in processes, on different machines
...) but that should be as transparent as possible.

The only thing asked to the developer is to either write "pure" functions to
process data (create a new dict, don't change in place, etc.), and everything
should be fine from this point.

It's a young rewrite of an old python2.7 tool that ran millions of
transformations per day for years on production, so as though it may not yet
be complete or fully stable (please, allow us to reach 1.0), the underlying
concepts work.

* Documentation: http://docs.bonobo-project.org/
* Release announcements: http://eepurl.com/csHFKL
* Old project (for reference, don't use anymore, instead, help us recode the missing parts in bonobo): http://etl.rdc.li/


.. image:: https://travis-ci.org/python-bonobo/bonobo.svg?branch=0.2
:target: https://travis-ci.org/python-bonobo/bonobo
:alt: Continuous Integration

.. image:: https://landscape.io/github/python-bonobo/bonobo/0.2/landscape.svg?style=flat
:target: https://landscape.io/github/python-bonobo/bonobo/0.2
:alt: Code Health

.. image:: https://img.shields.io/coveralls/python-bonobo/bonobo.svg
:target: https://coveralls.io/github/python-bonobo/bonobo?branch=0.2
:alt: Coverage

.. image:: https://readthedocs.org/projects/bonobo/badge/?version=0.2
:target: http://docs.bonobo-project.org/
:alt: Documentation

.. image:: https://img.shields.io/github/downloads/python-bonobo/bonobo/total.svg
:target: https://github.com/python-bonobo/bonobo/releases
:alt: Downloads

.. image:: https://img.shields.io/pypi/dm/bonobo.svg
:target: https://pypi.python.org/pypi/bonobo
:alt: Python Package on PyPI

----

Made with ♥ by `Romain Dorgueil <https://twitter.com/rdorgueil>`_ and `contributors <https://github.com/python-bonobo/bonobo/graphs/contributors>`_.

----

Roadmap (in progress)
:::::::::::::::::::::

Bonobo is young. This roadmap is alive, and will evolve. Its only purpose is to
write down incoming things somewhere.

Version 0.2
-----------

* Changelog
* Migration guide
* Update documentation
* Threaded does not terminate anymore (fixed ?)
* More tests

Bugs:

- KeyboardInterrupt does not work anymore. (fixed ?)
- ThreadPool does not stop anymore. (fiexd ?)

Configuration
.............

* Support for position arguments (options), required options are good candidates.

Context processors
..................

* Be careful with order, especially with python 3.5. (done)
* @contextual decorator is not clean enough. Once the behavior is right, find a
way to use regular inheritance, without meta.
* ValueHolder API not clean. Find a better way.

Random thoughts and things to do
................................

* Class-tree for Graph and Nodes

* Class-tree for execution contexts:

* GraphExecutionContext
* NodeExecutionContext
* PluginExecutionContext

* Class-tree for ExecutionStrategies

* NaiveStrategy
* PoolExecutionStrategy
* ThreadPoolExecutionStrategy
* ProcesPoolExecutionStrategy
* ThreadExecutionStrategy
* ProcessExecutionStrategy

* Class-tree for bags

* Bag
* ErrorBag
* InheritingBag

* Co-routines: for unordered, or even ordered but long io.

* "context processors": replace initialize/finalize by a generator that yields only once


* "execute" function:

.. code-block:: python

def execute(graph: Graph, *, strategy: ExecutionStrategy, plugins: List[Plugin]) -> Execution:
pass

* Handling console. Can we use a queue, and replace stdout / stderr ?

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

bonobo-0.2.0.macosx-10.12-x86_64.tar.gz (121.3 kB view details)

Uploaded Source

Built Distributions

bonobo-0.2.0-py3.6.egg (162.6 kB view details)

Uploaded Egg

bonobo-0.2.0-py2.py3-none-any.whl (111.6 kB view details)

Uploaded Python 2Python 3

File details

Details for the file bonobo-0.2.0.macosx-10.12-x86_64.tar.gz.

File metadata

File hashes

Hashes for bonobo-0.2.0.macosx-10.12-x86_64.tar.gz
Algorithm Hash digest
SHA256 0c8c87fcd179af7667c427794a73e035638a9507ee80a9139558b97a72cc235f
MD5 7e529a05ba0e9663b117a75c65fecebe
BLAKE2b-256 cdfc1eda5d6afc898f735a5e1bdfd01036e36aff7b8c9d4eff202468c40c7b8a

See more details on using hashes here.

File details

Details for the file bonobo-0.2.0-py3.6.egg.

File metadata

  • Download URL: bonobo-0.2.0-py3.6.egg
  • Upload date:
  • Size: 162.6 kB
  • Tags: Egg
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for bonobo-0.2.0-py3.6.egg
Algorithm Hash digest
SHA256 1f92bdcb198759add49a9c1c5623d7df5ebf4d83c1372a2be0ae9b6de78d9f22
MD5 3c32e38cd2ff1a5f611bd2dfff760def
BLAKE2b-256 a58b5c3da16e01fb90be68f037ff422f22c42119c59e09ea41efdd2e707a7561

See more details on using hashes here.

File details

Details for the file bonobo-0.2.0-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for bonobo-0.2.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 a8d3e937b0f475456139a390fe99aba9b251489900c35be69c62781c1c2e03f9
MD5 d66097add5fcda06cbf6df8a02d56956
BLAKE2b-256 f065b87eae10fff30be8c7091b976dbb2a94f556f6d5cf5354ff86e65c239c95

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page