Skip to main content

Python Pachyderm Client

Project description

# Python Pachyderm

[![PyPI Package latest releasee](https://img.shields.io/pypi/v/python-pachyderm.svg)](https://pypi.python.org/pypi/python-pachyderm)
[![Supported versions](https://img.shields.io/pypi/pyversions/python-pachyderm.svg)](https://pypi.python.org/pypi/python-pachyderm)
[![Slack Status](http://slack.pachyderm.io/badge.svg)](http://slack.pachyderm.io)

Official Python Pachyderm client. Created by [kalugny](https://github.com/kalugny) (formerly kalugny/pypachy.)

See the [API docs](https://htmlpreview.github.io/?https://github.com/pachyderm/python-pachyderm/blob/master/docs/python_pachyderm/index.html). Most of values are auto-generated from protobufs. It's generally easier to rely on the higher-level classes if they provide the functionality you need:
* [PfsClient](https://htmlpreview.github.io/?https://raw.githubusercontent.com/pachyderm/python-pachyderm/master/docs/python_pachyderm/pfs_client.m.html#python_pachyderm.pfs_client.PfsClient)
* [PpsClient](https://htmlpreview.github.io/?https://raw.githubusercontent.com/pachyderm/python-pachyderm/master/docs/python_pachyderm/pps_client.m.html#python_pachyderm.pps_client.PpsClient)

## Installation

```bash
pip install python-pachyderm
```

## Usage and options

All of the PFS functions used in `pachctl` are supported (almost) as-is.

There are some helper functions that help make things more pythonic:

* `commit` which is a context manager wrapper for `start_commit` and `finish_commit`
* `get_files` which supports getting the data from multiple files

### Naming commits

All functions that accept a `commit` argument will accept a tuple of `(repo, branch)` or `(repo, commit_id)`,
a string like `repo/branch` or `repo/commit_id` and a Commit object.

e.g:

```python
client.list_file(('my_repo', 'branch'), '/') # tuple
client.list_file('my_repo/commit_id', '/') # string
c = client.list_commit('my_repo')[0] # get some commit
client.list_file(c, '/') # and use it directly
```

### Basic usage example

```python
import python_pachyderm
client = python_pachyderm.PfsClient()
client.create_repo('test')
with client.commit('test', 'master') as c:
client.put_file_bytes(c, '/dir_a/data', b'DATA')
client.put_file_url(c, '/dir_b/icon.png', 'http://www.pearl-guide.com/forum/images/smilies/biggrin.png')

client.get_files('test/master', '/', recursive=True)
```

As of version 0.1.4, there is also limited support for PPS:

```python
pps_client = python_pachyderm.PpsClient()
pps_client.list_pipeline()
...
```

## Contributing

This driver is co-maintained by Pachyderm and the community. If you're looking to contribute to the project, this is a fantastic place to get involved.

### Getting started

To run tests, clone the repo, then run:

```
make init
tox
```


Changelog
=========

Unreleased
----------

* Added: Test automation tools pytest, tox, and Travis CI.
* Added: Minikube and Pachyderm deployment to Travis CI config.
* Fixed: Python 2.7 incompability issues.
* Changed: Switched docs to reStructuredText.
* Added: Separate CHANGELOG file.
* Added: PfsClient tests for init, list_repo, create_repo, delete_repo, start_commit, finish_commit, and commit methods.
* Changed: Modified PfsClient() delete_repo method error handling to match Go client behavior.
* Changed: Modified PfsClient() start_commit and commit methods to make the branch argument optional.
* Added: Bumpversion for tagging releases and semantic versioning.
* Fixed: PfsClient() initialization ignored pachd host and port environment variables.

0.1.5 (2017-08-06)
------------------

* Fixed: Miscellaneous bugs.

0.1.4 (2017-08-06)
------------------

* Added: Alpha support for PPS.
* Changed: Adapted for Pachyderm ``1.5.2``.

0.1.3 (2017-05-18)
------------------

* Fixed: ``inspect_commit`` was broken.
* Added: ``provenances_for_repo`` function gives all the provenances for the commits in the repo.

0.1.2 (2017-04-20)
------------------

* Added: ``PfsClient`` default parameters now use the environment variables for pachd.
* Added: ``put_file_bytes`` can accept an iterator.
* Changed: ``commit`` now tries to close the commit if an error occurred inside the context.
* Added: More examples and a changelog to README.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

python-pachyderm-1.8.5.post3.tar.gz (70.7 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page