Archive (safe)-Removal Rolling Toolbox
Project description
A2R
The name a2r, pronounced as a double r, stands for Archive safe-Remove Rolling. It's a toolbox that provides different tool for:
- Archive automatically Oceanic experiment outputs base on yaml configuration file
- (safe) Rm files if a local copy already exists
- Automatic Rolling with different options on multiple directory with a single command. Also in this case the info about what to delete and how are provided using a yaml configuration file
Installation
Via pip
pip install a2r
Via conda/mamba
mamba install a2r
Usage
Usage: a2r [OPTIONS] COMMAND [ARGS]...
Options:
--version Show the version and exit.
--help Show this message and exit.
Commands:
rolling
saferm
update-md5
Rolling
To keep a directory under rolling, use the command:
Usage: cli.py rolling [OPTIONS]
Options:
-c, --config PATH [required]
--dry-run Dry run mode.
-d, --debug Enable debug mode.
--help Show this message and exit.
Rolling Configuration File
Here a template of a rolling configuration file:
- !CleanPath
path: $PATH_UNDER_ROLLING
fmt: ???????? # YYYYMMDD
safe:
to_keep: X
reference_paths:
- <REF1>
- <REF2>
conditional:
to_keep: Y
expected_files:
- file.exe
- tmp.nc
force:
to_keep: Z
let's comment each section.
CleanPath object
With this, we start to declare a list of !CleanPath objects with two mandatory attributes:
- !CleanPath
path: $PATH_UNDER_ROLLING
fmt: ???????? # YYYYMMDD
path
: it's the path to keep under rollingfmt
: it's a string bash that represent the format of file/dir to delete. It can contains jolly characters?
or*
Safe
safe:
to_keep: X
reference_paths:
- <REF1>
- <REF2>
safe
: means for safe rolling, which means delete a file only if an identical local copy already existsto_keep
: how much dir/file to not include in the rollingreference_paths
: a list of path where to find if a local copy already exists
To consider that the safe mode doesn't remove the dir under rolling.
Conditional
#### Conditional conditional:
to_keep: Y
expected_files:
- file.exe
- tmp.nc
conditional
: specify to remove a dir if some conditions are meetto_keep
: how much dir/file to not include in the rollingexpected_files
: specify the exact list of files expected to find in rolling path to trigger the rm operations. The filename can contains jolly character?
and*
Force
force: # optional
to_keep: Z
force
: Enable path rm without any checkto_keep
: how much dir/file to not include in the rolling
Update md5 dir
A requirement to safe clean, is to compute in advance the md5 of all files in the rolling path using the command:
update-md5 [OPTIONS]
Options:
-p, --path PATH [required]
-d, --debug Enable debug mode.
--help Show this message and exit.
The command will create in the directory a file called .dir_md5.txt
with the following structure:
md5hash filename1
md5hash filename2
Safe Clean
Safe Rm command is the equivalent of safe rm rolling section: it removes files from clean path only if exists an identical copy in keep path
Usage: cli.py saferm [OPTIONS]
Options:
--keep PATH [required]
--clean PATH [required]
-f, --force Delete files without confirm request
-d, --dry-run Disable file removal
-d, --debug Enable debug mode.
--help Show this message and exit.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file a2r-0.1.0.tar.gz
.
File metadata
- Download URL: a2r-0.1.0.tar.gz
- Upload date:
- Size: 14.8 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.8.3 CPython/3.10.14 Linux/6.5.0-1019-oem
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | b9a78e82bcd754b31b4477454ba58e7a7b264b11e9b2cd3e286eebe321a1d2fe |
|
MD5 | 07096ad79b0092c59971ba2dc037fc94 |
|
BLAKE2b-256 | 8cc654923fff01c54f59b125c35c678325e3f0076ef43bf436151cadd23171ed |
File details
Details for the file a2r-0.1.0-py3-none-any.whl
.
File metadata
- Download URL: a2r-0.1.0-py3-none-any.whl
- Upload date:
- Size: 18.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.8.3 CPython/3.10.14 Linux/6.5.0-1019-oem
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 47f2832f318624c35ae877426b0ace05f987f087dfbe15cbc6bd3521cacff5c7 |
|
MD5 | 1f0ecaf5bf436e8ff059f6088b1d625c |
|
BLAKE2b-256 | 94ff61d3d0339f96dd701fb899760f6b18194725d5d44bf405906aeedca07e8e |