hydra-filter-sweeper 2.1.0

A Hydra plugin to extend the basic sweeper with customizable filters.

Hydra Filter Sweeper Plugin

hydra-filter-sweeper is a plugin for Hydra that extends the basic sweeper by the addition of filters, enabling more targeted parameter sweeps.

The plugin is compatible with any Hydra launcher plugin. The minimum required Hydra version is 1.3.2.

Features

Customizable Filters

• Apply expressions, existence checks, or custom filter classes to the sweep.

Flexible Filter Conditions

• Include fail-safe conditions to gracefully handle possible exceptions or suppress log messages.

Interpolation Support

• Utilize OmegaConf's interpolation syntax to reference configuration values.

Installation

Install hydra-filter-sweeper via pip:

pip install hydra-filter-sweeper

Usage

To use hydra-filter-sweeper, override the default sweeper with filter at the end of the defaults list.

Filters are specified as a list of configurations under the hydra/sweeper/filters key.

If any filter evaluates to True, the current configuration is excluded from the sweep.

If no filters list is provided or all filters evaluate to False, all configurations are included and the sweeper resembles the default behavior of Hydra's basic sweeper.

Example Configuration

defaults:
  - _self_
  - override hydra/sweeper: filter

some_value: four

hydra:
  mode: MULTIRUN
  sweeper:
    params:
      +foo: 1,2,3
      +bar: one, two, three
    filters:
      - hydra_filter_sweeper.Expression:
          expr: foo == 1 and bar == "two"
      - hydra_filter_sweeper.Exists:
          path: some_directory/some.file
      - some_custom_filter.SomeFilter:
          some_arg: ${some_value}

Filters

Expression Filter

Filter a configuration based on the evaluation of a Python expression using the configuration as context.

All keys in the configuration are added to the attributes list of the evaluation model in addition to lists, tuples, and safe function calls (e.g., str, int, float, len).

The configuration is excluded if the expression evaluates to True.

Parameters:

• expr (str): The Python expression to be evaluated.

Example Configurations

hydra/sweeper/filters:
  - hydra_filter_sweeper.Expression:
      expr: foo == 1 and bar == "two"
  - hydra_filter_sweeper.Expression:
      expr: bar == ${some_value}
  - hydra_filter_sweeper.Expression:
      expr: undefined == 1 and bar == "two"
      _fail_: false
      _log_: false

Exists Filter

Filter a configuration based on the existence of a file or directory relative to the job output directory.

The configuration is excluded if the file or directory exists.

Parameters:

• path (str): The path to the file or directory to check for existence.

Example Configurations

hydra/sweeper/filters:
  - hydra_filter_sweeper.Exists:
      path: some_directory/some.file
  - hydra_filter_sweeper.Exists:
      path: some_directory
      _log_: false
  - hydra_filter_sweeper.Exists:
      path: some_directory/${some_value}.file
  - hydra_filter_sweeper.Exists:
      path: null
      _fail_: false

Custom Filters

Filter a configuration using a custom filter class that implements the hydra_filter_sweeper.AbstractFilter interface. The filter class should be specified by its Python relative import path.

The the filter method is called with any additional keyword arguments provided in the configuration.

The optional reason method can return an explanation for filtering the provided configuration and receives the same arguments as filter.

Example Configurations

from hydra_filter_sweeper import AbstractFilter


class SomeFilter(AbstractFilter):
    def filter(self, some_arg: str) -> bool:
        return some_arg == "expected_value"

    def reason(self, some_arg: str) -> str:
        return f"Filtered because 'some_arg' matched the expected value!"

class WithoutArguments(AbstractFilter):
    def filter(self) -> bool:
        return self.config.some_value == "expected_value"

class AnotherFilter(AbstractFilter):
    def filter(self, first_arg: str, second_arg: str) -> bool:
        return first_arg == "expected_value" and second_arg == "another_expected_value"

hydra/sweeper/filters:
  - some_custom_filter.SomeFilter:
      some_arg: ${some_value}
  - some_custom_filter.WithoutArguments
  - some_custom_filter.AnotherFilter:
      first_arg: ${some_value}
      second_arg: ${another_value}
      _fail_: false
      _log_: false

Fail and Log Options

For all filters, the _fail_ and _log_ options can be used to control the behaviour of the filter:

• _fail_ (bool, default: True): If True, the filter will raise an exception if it fails to evaluate. If False, the filter will not raise an exception, and the configuration will be included in the sweep.
• _log_ (bool, default: True): If True, the filter will log a message when it is applied. If False, no log message is logged.

Contributing

Contributions are welcome! For bug reports or requests, please submit an issue.

To contribute, please fork the repository and submit a pull request.

We use uv for dependency management, Ruff for code formatting, codespell for spell checking, pytest for testing, and pre-commit for managing the hooks.

Install the development dependencies with:

uv sync
pre-commit install

Both formatting and spell checking are enforced by pre-commit hooks.

We strive for 100% test coverage. To run the tests locally, use:

pytest