Skip to main content

Python Pachyderm Client

Project description

Python Pachyderm

Official Python Pachyderm Client

Created by [kalugny](

(Formerly kalugny/pypachy)

Python Pachyderm Client

A python client wrapper for the Pachyderm_ API.


* Currently implements the PFS interface and only alpha support for the PPS interface.

* Supports Pachyderm versions 1.4 and up.


.. code:: bash

pip install pypachy

Usage and options

All of the PFS functions used in ``pachctl`` are supported (almost) as-is.

There are some helper functions that help make things more pythonic:

* ``commit`` which is a context manager wrapper for ``start_commit`` and ``finish_commit``
* ``get_files`` which supports getting the data from multiple files

Naming commits

All functions that accept a ``commit`` argument will accept a tuple of ``(repo, branch)`` or ``(repo, commit_id)``,
a string like ``repo/branch`` or ``repo/commit_id`` and a Commit object.


.. code:: python

>>> client.list_file(('my_repo', 'branch'), '/') # tuple
>>> client.list_file('my_repo/commit_id', '/') # string
>>> c = client.list_commit('my_repo')[0] # get some commit
>>> client.list_file(c, '/') # and use it directly

Basic usage example

.. code:: python

>>> import pypachy

>>> client = pypachy.PfsClient()
>>> client.create_repo('test')
>>> with client.commit('test', 'master') as c:
...: client.put_file_bytes(c, '/dir_a/data', b'DATA')
...: client.put_file_url(c, '/dir_b/icon.png', '')
>>> client.get_files('test/master', '/', recursive=True)
{'/dir_a/data': b'DATA',
'/dir_b/icon.png': b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\x10\x00\x00\x00\x10\x08...'}

As of version 0.1.4, there is also limited support for PPS:

.. code:: python

>>> pps_client = pypachy.PpsClient()
>>> pps_client.list_pipeline()

To Do

* Achieve full test coverage for PFS and PPS.
* Add support for ``description`` field in ``Commit``.
* Add support for ``ObjectAPI``


See `CHANGELOG.rst <>`_.

.. _Pachyderm:



* Added: Test automation tools pytest, tox, and Travis CI.
* Added: Minikube and Pachyderm deployment to Travis CI config.
* Fixed: Python 2.7 incompability issues.
* Changed: Switched docs to reStructuredText.
* Added: Separate CHANGELOG file.
* Added: PfsClient tests for init, list_repo, create_repo, delete_repo, start_commit, finish_commit, and commit methods.
* Changed: Modified PfsClient() delete_repo method error handling to match Go client behavior.
* Changed: Modified PfsClient() start_commit and commit methods to make the branch argument optional.
* Added: Bumpversion for tagging releases and semantic versioning.
* Fixed: PfsClient() initialization ignored pachd host and port environment variables.

0.1.5 (2017-08-06)

* Fixed: Miscellaneous bugs.

0.1.4 (2017-08-06)

* Added: Alpha support for PPS.
* Changed: Adapted for Pachyderm ``1.5.2``.

0.1.3 (2017-05-18)

* Fixed: ``inspect_commit`` was broken.
* Added: ``provenances_for_repo`` function gives all the provenances for the commits in the repo.

0.1.2 (2017-04-20)

* Added: ``PfsClient`` default parameters now use the environment variables for pachd.
* Added: ``put_file_bytes`` can accept an iterator.
* Changed: ``commit`` now tries to close the commit if an error occurred inside the context.
* Added: More examples and a changelog to README.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
python-pachyderm-1.6.8.post8.tar.gz (56.7 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page