alyx-registrator 0.0.8

Utilitaries to easily register files to an alyx-database/ONE environment.

Documentation for rules.json in one_registrator Package

The rules.json file is used to define rules for processing and managing files in the one_registrator package. This file contains patterns, rules, and configurations that dictate how files should be renamed, included, excluded, or deleted based on specific conditions.

Structure of rules.json

1. re_patterns

• Description: Defines regular expression patterns used in rules for matching file attributes during renaming or matching conditions.
• Format: A dictionary where keys are pattern names and values are regex strings.

2. rules

• Description: Contains a set of rules that define conditions and actions for file processing.
• Format: A dictionary where each key is a rule name and the value is a dictionary with the following keys:
  • if: Conditions that must be met for the rule to apply.
  • on: Actions to take when conditions are met.
  • rename: Specifies how to rename files if one of the Actions for this rule is a rename action.
  • overrides: (Optional) List of other rules that this rule can override, in condition that several rules match the same element.
    Note that if several rules match the same element and the ovveriding is not specified in a way that a single rule can be identified as the single rule to apply to this element, it will conflct and nothing will happend for this element (an error will be thrown in the results table)

3. excluded_folders

• Description: Lists folder names to be excluded from processing.
• Format: An array of folder names.

4. excluded_filenames

• Description: Lists filenames to be excluded from processing.
• Format: An array of filenames.

5. cleanup_folders

• Description: Lists folders to be cleaned up if they become empty after processing.
• Format: An array of folder names.

rules structure

Conditions (if)

• Elements: Parts of a filename or file path that can be used to define conditions.

  • source_path: The original path of the file.
  • subject: The subject associated with the file.
  • date: The date associated with the file.
  • number: A number associated with the file.
  • root: The root directory of the file.
  • object: The object part of the filename.
  • attribute: The attribute part of the filename.
  • extension: The file extension.
  • extra: Additional information or metadata.
  • collection: The collection or category the file belongs to.
  • revision: The revision number or version of the file.

• Operations: Define how the conditions are evaluated.

  • exact: Matches the element exactly.
  • contain: Checks if the element contains a specified value.
  • match: Uses regex to match the element.
  • exact_not: Matches if the element does not exactly match the specified value.
  • contain_not: Matches if the element does not contain the specified value.

• Example:

  "if": {
      "extension": {
          "exact": ["tif", "tiff"]
      },
      "collection": {
          "contain": "imaging_data"
      }
  }
  

Actions (on)

• Triggers: Conditions under which actions are executed.

  • match: Triggered when the rule conditions are met.
  • destination_exists: Triggered if the destination file already exists.
  • rename_unchanged: Triggered if the rename operation does not change the filename.
  • rename_error: Triggered if there is an error during the rename operation.

• Actions: Operations to perform when triggers are activated.

  • rename: Renames the file according to the specified rules.
  • include: Includes the file in further processing.
  • delete: Deletes the file.
  • exclude: Excludes the file from further processing.
  • abort: Stops processing due to an error or conflict.

• Example:

"on": {
    "match": "rename",
    "destination_exists": "abort",
    "rename_unchanged": "exclude"
}

Rename (rename)

• Elements: object, attribute, collection, extra.
• Example:

"rename": {
    "object": "imaging",
    "attribute": "frames",
    "extra": {
        "pattern": "TIFF_FRAME_NO",
        "eval": "match[1].zfill(5)"
    }
}

Example rules.jon file

{
    "re_patterns": {
        "VGAT_MOUSE_NO": "Vgat.*?(\\d{1,3}).*jRGECO",
        "TIFF_FRAME_NO": ".*(\\d{5})\\.tiff?$"
    },
    "rules": {
        "imaging_data_rule1": {
            "if": {
                "extension": {
                    "exact": ["tif", "tiff"]
                },
                "collection": {
                    "contain": "imaging_data"
                }
            },
            "on": {
                "match": "rename",
                "destination_exists": "abort",
                "rename_unchanged": "exclude"
            },
            "rename": {
                "object": "imaging",
                "attribute": "frames",
                "extra": {
                    "pattern": "TIFF_FRAME_NO",
                    "eval": "match[1].zfill(5)"
                }
            }
        }
    },
    "excluded_folders": ["figures", "trash"],
    "excluded_filenames": ["session_meta.json"],
    "cleanup_folders": ["pupil", "whiskers"]
}