mldatafind 0.1.8

Luigi/Law Tasks for streamlining gravitational wave data discovery

mldatafind

Law workflows for streamling gravitational wave data discovery for ML applications

Example

To run the example configuration, first build the container to your desired location

export CONTAINER_PATH=/path/to/mldatafind.sif
apptainer build $CONTAINER_PATH apptainer.def

Next, the Fetch task, which will query science segments and strain data, can be run using local resources

LAW_CONFIG_FILE=./example.cfg uv run law run mldatafind.law.tasks.Fetch --workflow local --local-scheduler --sandbox mldatafind::$CONTAINER_PATH

If you're on a machine with condor access like the LDG, the Fetch task can also trivially utilize condor resources by setting --workflow htcondor

LAW_CONFIG_FILE=./example.cfg uv run law run mldatafind.law.tasks.Fetch --workflow htcondor --local-scheduler --sandbox mldatafind::$CONTAINER_PATH

condor log files will be stored under the condor_directory argument of the Fetch task