Housekeeper takes care of files
Project description
Housekeeper
Store, tag, fetch, and archive files with ease 🗃
Housekeeper is a tool that aims to provide:
- a backend for storing versioned bundles of files
- different interfaces (Python, CLI, REST) for fetching files based on tags
- a way to backup and retrieve bundles from long-term storage
Todo
- re-implement the archive/encryption interface [@ingkebil]
- handle clean up of expired bundles [@robinandeer]
- expand the CLI with
get
command etc. [@robinandeer]
Installation
Housekeeper written in Python 3.6+ and is available on the Python Package Index (PyPI).
pip install housekeeper
If you would like to install the latest development version:
git clone https://github.com/Clinical-Genomics/housekeeper
cd housekeeper
pip install --editable .
Contributing
Housekeeper is using github flow branching model as described in our development manual.
Documentation
Command line interface
Config file
Housekeeper supports a very simple YAML config. The following options are supported:
---
database: mysql+pymysql://userName:passWord@domain.com/database
root: /path/to/root/dir
The root
option is used to store files within the Housekeeper context.
Command: init
Setup (or reset) the database. It will simply setup all the tables in the database. You can reset an existing database by using the --reset
option.
housekeeper --database "sqlite:///hk.sqlite3" init
Success! New tables: bundle, file, file_tag_link, tag, version
Command: include
Include (hard-link) all files of an existing bundle version into Housekeeper and the root
path.
housekeeper myBundle
This will only work if the bundle only has a single version which can be "imported". If you want to import a specific version of a bundle you can use the --version
option.
Command: delete files
Delete files that are not on disk anymore like his:
housekeeper delete files --tag fastq --notondisk
Remove all bam files before a certain date:
housekeeper delete files --tag bam --before 2017-06-15
Remove fastq files from a flowcell:
housekeeper delete files --tag fastq --tag H0HKKALXX
It'll always ask for confirmation, unless you add --yes:
housekeeper delete files --bundle sillyfish --yes
If you do not provide a --tag or --bundle, essentially deleting everything, the function will not let you do that.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for housekeeper-3.0.1-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a6a3e91e8490e4e66c1c3d5845c9880863c6c0227524edfc27d101178e973f8e |
|
MD5 | 0d7d5b9a3063aee24469f0cc7891d9eb |
|
BLAKE2b-256 | bdc41994fedaeb67d7935d629ecf1d853977d75b2373f5892e580c1af3629ab7 |