Skip to main content

File data orchestrator

Project description

Housekeeper

Housekeeper tests Coverage Status CodeFactor Code style: black

Store, tag, fetch, and archive files with ease 🗃

Housekeeper is a tool that aims to provide:

  • a backend for storing versioned bundles of files
  • different interfaces (Python, CLI, REST) for fetching files based on tags
  • a way to backup and retrieve bundles from long-term storage

Installation

Housekeeper written in Python 3.6+ and is available on the Python Package Index (PyPI).

poetry install

If you would like to install the latest development version:

git clone https://github.com/Clinical-Genomics/housekeeper
cd housekeeper
poetry install

Contributing

Housekeeper is using GitHub flow branching model as described in our development manual.

Documentation

Command line interface

Config file

Housekeeper supports a basic YAML config. The following options are supported:

---
database: mysql+pymysql://userName:passWord@domain.com/database
root: /path/to/root/dir

The root option is used to store files within the Housekeeper context.

Command: init

Setup (or reset) the database. It will simply setup all the tables in the database. You can reset an existing database by using the --reset option.

housekeeper --database "sqlite:///hk.sqlite3" init
Success! New tables: bundle, file, file_tag_link, tag, version

Command: include

Include (hard-link) all files of an existing bundle version into Housekeeper and the root path.

housekeeper myBundle

This will only work if the bundle only has a single version which can be "imported". If you want to import a specific version of a bundle you can use the --version option.

Command: delete files

Delete files that are not on disk anymore like his: housekeeper delete files --tag fastq --notondisk

Remove all bam files before a certain date: housekeeper delete files --tag bam --before 2017-06-15

Remove fastq files from a flowcell: housekeeper delete files --tag fastq --tag H0HKKALXX

It'll always ask for confirmation, unless you add --yes: housekeeper delete files --bundle sillyfish --yes

If you do not provide a --tag or --bundle, essentially deleting everything, the function will not let you do that.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

housekeeper-4.13.13.tar.gz (44.2 kB view details)

Uploaded Source

Built Distribution

housekeeper-4.13.13-py3-none-any.whl (34.6 kB view details)

Uploaded Python 3

File details

Details for the file housekeeper-4.13.13.tar.gz.

File metadata

  • Download URL: housekeeper-4.13.13.tar.gz
  • Upload date:
  • Size: 44.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/2.1.2 CPython/3.12.10 Linux/6.11.0-1012-azure

File hashes

Hashes for housekeeper-4.13.13.tar.gz
Algorithm Hash digest
SHA256 047cdf5f901d8de7ac8ffa0a545fd39f52800b0ca6f2dad7a49e32b071f30d8f
MD5 5351ce7c52535be6567e66a8c2172321
BLAKE2b-256 49183f9d8ace953cabab2e8a625cec9a34ddb7acd50f501f4556df0e3a4cacb2

See more details on using hashes here.

File details

Details for the file housekeeper-4.13.13-py3-none-any.whl.

File metadata

  • Download URL: housekeeper-4.13.13-py3-none-any.whl
  • Upload date:
  • Size: 34.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/2.1.2 CPython/3.12.10 Linux/6.11.0-1012-azure

File hashes

Hashes for housekeeper-4.13.13-py3-none-any.whl
Algorithm Hash digest
SHA256 0a2f4c2cae8dec01e5f9a4232d4e199a2d994adb205d7659638b949f88457943
MD5 b5971e71b3d5d55676e1190ebbc0b763
BLAKE2b-256 fbe7592ba29d4a2987186c5543381eb132468639558100adb693fb4bb5a4671c

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page