Skip to main content

Simple data archiving tool

Project description

arx - a simple data archiving tool

'arx' is a simple data archiving tool. A local directory, or "repo", is initialized (with the 'init' command) to be a staging ground and mirror of a remote archive. Files and directories can be copied into the repo and then committed to the remote archive (with the 'commit' command). Files and directories in the remote archive can be checked out from the archive for viewing and processing (with the 'checkout' command).

Arx assumes a repository exists as tree of files on a remote server accessible via ssh and rsync. No other remote configuration is needed.

NOTE: Arx maintains only a write-once archive. Once a file has been committed to the repository it can't be changed or updated.



arx only depends on one python package that's not included in the standard library (pyyaml). It otherwise depends on a couple of non-python packages that are usually available by default in most systems:

  • ssh client
  • rsync
  • find

It is unlikely on most systems that these apps wouldn't be available.


command line interface

Assuming an existing pre-configured remote of the form HOST:PATH, initialize a local repo mirror:

$ arx init HOST:PATH my_repo
$ cd my_repo
$ arx config
root: /path/to/my_repo
remote: HOST:PATH

Add a file to the repo:

$ cp /path/to/other_file file2

List files in the archive and in the local repo:

$ arx list
- file1
+ file2

The '-'/'+' prefixes indicate files that only exist remotely/locally, respectively. No prefix indicates the file exists in both places.

Checkout a file from the archive:

$ arx checkout file1
$ arx list
+ file2

Commit a file to the repo:

$ arx commit file2
$ arx list

python library

Arx also includes a python library from which all of the same functionality can be accessed.

The basic interface is through the Repo class. A new repo can be created with the Repo.init method:

import arx

repo = arx.Repo.init(HOST:PATH, '/path/to/my_repo')

You can then use the checkout, commit, and list methods:

repo = arx.Repo('/path/to/my_repo')
for f in repo.list(remote=True):
for f in repo.list():

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

arx-0.1.0.tar.gz (31.5 kB view hashes)

Uploaded source

Built Distribution

arx-0.1.0-py3-none-any.whl (21.6 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page