Skip to main content

Housekeeper takes care of files

Project description

Housekeeper

Housekeeper tests Coverage Status CodeFactor Code style: black

Store, tag, fetch, and archive files with ease 🗃

Housekeeper is a tool that aims to provide:

  • a backend for storing versioned bundles of files
  • different interfaces (Python, CLI, REST) for fetching files based on tags
  • a way to backup and retrieve bundles from long-term storage

Todo

  • <input type="checkbox" disabled="" /> re-implement the archive/encryption interface [@ingkebil]
  • <input type="checkbox" disabled="" /> handle clean up of expired bundles [@robinandeer]
  • <input type="checkbox" disabled="" /> expand the CLI with get command etc. [@robinandeer]

Installation

Housekeeper written in Python 3.6+ and is available on the Python Package Index (PyPI).

pip install housekeeper

If you would like to install the latest development version:

git clone https://github.com/Clinical-Genomics/housekeeper
cd housekeeper
pip install --editable .

Contributing

Housekeeper is using github flow branching model as described in our development manual.

Documentation

Command line interface

Config file

Housekeeper supports a very simple YAML config. The following options are supported:

---
database: mysql+pymysql://userName:passWord@domain.com/database
root: /path/to/root/dir

The root option is used to store files within the Housekeeper context.

Command: init

Setup (or reset) the database. It will simply setup all the tables in the database. You can reset an existing database by using the --reset option.

housekeeper --database "sqlite:///hk.sqlite3" init
Success! New tables: bundle, file, file_tag_link, tag, version

Command: include

Include (hard-link) all files of an existing bundle version into Housekeeper and the root path.

housekeeper myBundle

This will only work if the bundle only has a single version which can be "imported". If you want to import a specific version of a bundle you can use the --version option.

Command: delete files

Delete files that are not on disk anymore like his: housekeeper delete files --tag fastq --notondisk

Remove all bam files before a certain date: housekeeper delete files --tag bam --before 2017-06-15

Remove fastq files from a flowcell: housekeeper delete files --tag fastq --tag H0HKKALXX

It'll always ask for confirmation, unless you add --yes: housekeeper delete files --bundle sillyfish --yes

If you do not provide a --tag or --bundle, essentially deleting everything, the function will not let you do that.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for housekeeper, version 3.0.0
Filename, size File type Python version Upload date Hashes
Filename, size housekeeper-3.0.0-py2.py3-none-any.whl (22.3 kB) File type Wheel Python version py2.py3 Upload date Hashes View
Filename, size housekeeper-3.0.0.tar.gz (32.0 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page