Skip to main content

Python Pachyderm Client

Project description

A python client wrapper for Pachyderm <https://www.pachyderm.io/> API.

NOTES

  1. Currently implements only the PFS interface.

  2. Supports Pachyderm versions 1.4 and up

Installing

$ pip install pypachy

Instructions

All of the PFS functions used in pachctl are supported (almost) as-is. There are some helper functions that help make things more pythonic: * commit which is a context manager wrapper for start_commit and finish_commit * get_files which supports getting the data from multiple files

Naming commits

All functions that accept a commit argument will accept a tuple of (repo, branch) or (repo, commit_id), a string like repo/branch or repo/commit_id and a Commit object.

e.g:

>>> client.list_file(('my_repo', 'branch'), '/')    # tuple
>>> client.list_file('my_repo/commit_id', '/')      # string
>>> c = client.list_commit('my_repo')[0]            # get some commit
>>> client.list_file(c, '/')                        # and use it directly

Basic usage example

>>> import pypachy

>>> client = pypachy.PfsClient()
>>> client.create_repo('test')
>>> with client.commit('test', 'master') as c:
...:     client.put_file_bytes(c, '/dir_a/data', b'DATA')
...:     client.put_file_url(c, '/dir_b/icon.png', 'http://www.pearl-guide.com/forum/images/smilies/biggrin.png')
...:
>>> client.get_files('test/master', '/', recursive=True)
{'/dir_a/data': b'DATA',
 '/dir_b/icon.png': b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\x10\x00\x00\x00\x10\x08...'}

As of version 0.1.4, there is also limited support for PPS:

>>> pps_client = pypachy.PpsClient()
>>> pps_client.list_pipeline()
...

TODO

  • Test, test, test!

  • Add support for ObjectAPI

Changelog

0.1.5 - Bug fixes

0.1.4 - Added: Alpha support for PPS - Changed: Adapted for Pachyderm 1.5.2

0.1.3 - Fixed: inspect_commit was broken - Added: provenances_for_repo function gives all the provenances for the commits in the repo

0.1.2 - Added: PfsClient default parameters now use the environment variables for pachd - Added: put_file_bytes can accept an iterator - Changed: commit now tries to close the commit if an error occurred inside the context - Added: More examples and a changelog to README

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pypachy-0.1.5.tar.gz (42.3 kB view details)

Uploaded Source

File details

Details for the file pypachy-0.1.5.tar.gz.

File metadata

  • Download URL: pypachy-0.1.5.tar.gz
  • Upload date:
  • Size: 42.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for pypachy-0.1.5.tar.gz
Algorithm Hash digest
SHA256 10bd7efb5def1b22ee329635a3db7731aa350c1538d9e29376dd46debe825361
MD5 f26ec7bf0be2dbd298f24390a03b004b
BLAKE2b-256 e7cbe0324c8bb05d82b5f939094214e36d318e279f6ea470fcf3298f8b35e90c

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page