rl-opts 0.2.0

Reinforcement Learning of Optimal Search strategies

RL-OptS

Reinforcement Learning of Optimal Search strategies

This library builds the necessary tools needed to study, replicate and develop reinforcement learning agents for target search problems, as well as a benchmark baselines with which to compare. This library is based in two different publications:

• “Optimal foraging strategies can be learned” by G. Muñoz-Gil, A. López-Incera, L. J. Fiderer and H. J. Briegel. Here we developed agents able to learn how to forage efficiently in environments with multiple targets.

• “Learning to reset in target search problems” by G. Muñoz-Gil, H. J. Briegel and M. Caraglio. Here we extended the agents to be able to reset to the origin, a feature that has revolutionize target search problems in the last years.

Installation

You can access all these tools installing the python package rl_opts via Pypi:

pip install rl-opts

You can also opt for cloning the source repository and executing the following on the parent folder you just cloned the repo:

pip install -e rl_opts

This will install both the library and the necessary packages.

Tutorials

We have prepared a series of tutorials to guide you through the most important functionalities of the package. You can find them in the Tutorials folder of the Github repository or in the Tutorials tab of our webpage, with notebooks that will help you navigate the package as well as reproducing the results of our paper via minimal examples. In particular, we have three tutorials:

• Reinforcement learning : shows how to train a RL agent based on Projective Simulation agents to search targets in randomly distributed environments as the ones considered in our paper.
• Learning to reset in target search problems : shows how to train a RL agent similar to the previous, but with the ability to reset to the origin, an action that is learned along its spatial dynamics.
• Imitation learning : shows how to train a RL agent to imitate the policy of an expert equipped with a pre-trained policy. The latter is based on the benchmark strategies common in the literature.
• Benchmarks : shows how launch various benchmark strategies with which to compare the trained RL agents.

Cite

We kindly ask you to cite our paper if any of the previous material was useful for your work:

@article{munoz2024optimal,
  title={Optimal foraging strategies can be learned},
  author={Mu{\~n}oz-Gil, Gorka and L{\'o}pez-Incera, Andrea and Fiderer, Lukas J and Briegel, Hans J},
  journal={New Journal of Physics},
  volume={26},
  number={1},
  pages={013010},
  year={2024},
  publisher={IOP Publishing}
}