A2R

The name a2r, pronounced as a double r, stands for Archive safe-Remove Rolling. It's a toolbox that provides different tool for:

• Archive automatically Oceanic experiment outputs base on yaml configuration file
• (safe) Rm files if a local copy already exists
• Automatic Rolling with different options on multiple directory with a single command. Also in this case the info about what to delete and how are provided using a yaml configuration file

Installation

Via pip

pip install a2r

Via conda/mamba

mamba install a2r

Usage

Usage: a2r [OPTIONS] COMMAND [ARGS]...

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  rolling
  saferm
  update-md5

Rolling

To keep a directory under rolling, use the command:

Usage: cli.py rolling [OPTIONS]

Options:
  -c, --config PATH  [required]
  --dry-run          Dry run mode.
  -d, --debug        Enable debug mode.
  --help             Show this message and exit.

Rolling Configuration File

Here a template of a rolling configuration file:

- !CleanPath
    path: $PATH_UNDER_ROLLING
    fmt: ????????   # YYYYMMDD
    safe:
        to_keep: X
        reference_paths:
            - <REF1>
            - <REF2>
    conditional:
        to_keep: Y
        expected_files:
            - file.exe
            - tmp.nc
    force:
        to_keep: Z

let's comment each section.

- !CleanPath
    path: $PATH_UNDER_ROLLING
    fmt: ????????   # YYYYMMDD

  safe:
      to_keep: X
      reference_paths:
          - <REF1>
          - <REF2>

  conditional:
      to_keep: Y
      expected_files:
          - file.exe
          - tmp.nc

    force: # optional
        to_keep: Z

Update md5 dir

A requirement to safe clean, is to compute in advance the md5 of all files in the rolling path using the command:

update-md5 [OPTIONS]

Options:
  -p, --path PATH  [required]
  -d, --debug      Enable debug mode.
  --help           Show this message and exit.

The command will create in the directory a file called .dir_md5.txt with the following structure:

md5hash filename1
md5hash filename2

Safe Clean

Safe Rm command is the equivalent of safe rm rolling section: it removes files from clean path only if exists an identical copy in keep path

Usage: cli.py saferm [OPTIONS]

Options:
  --keep PATH    [required]
  --clean PATH   [required]
  -f, --force    Delete files without confirm request
  -d, --dry-run  Disable file removal
  -d, --debug    Enable debug mode.
  --help         Show this message and exit.