academia-rl 0.2.0

Easy-to-use tools for Curriculum Learning

academia

This package’s purpose is to provide easy-to-use tools for Curriculum Learning. It is a part of an engineering thesis at Warsaw University of Technology that touches on the topic of curriculum learning.

Documentation

https://academia.readthedocs.io/

Experiments

As a part of our engineering thesis, numerous experiments on Curriculum Learning have been conducted using this package. Their code and results can be found in a separate repo.

Sources

An unordered list of interesting books and papers

Books

1. Reinforcement Learning: An Introduction (Barto, Sutton)

Papers

Paper Link	Short Description	Related Papers
Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey	Survey of curriculum learning papers. Formalises curriuclum learning based on a variety of attributes and gives a good introduction into the topic.
Curriculum Design for Machine Learners in Sequential Decision Tasks	Talks about curricula desgined by non-experts (i.e. people who do not know much/anything about a given domain). Uses "dog training" game as the basis of their experiments. Users can design a curriculum by sequencing any number of tasks in more or less complex environments. A target (more difficult) task is also provided to the user but they are not allowed to include it in the curriculum. A trainer model is used to go through the curriculum and provide feedback to the agent. They measure how good a curriculum is by the number of feedbacks that the trainer has to give to the agent i.e. if a curriculum is well designed the agent will require a relatively smaller number of feedbacks from the trainer to move on to the next task. They use three different trainer behaviours and show that the type of the trainer oes not influence the impact of curriuclum design i.e. if a curriculum is well designed under one trainer it is also well designed under another trainer. Another condition for a curriculum to be considered good is that the number of feedbacks over the entire curriculum (with the target task) should be smaller than the number of feedbacks when training on the target task alone. Results show that non-experts can design a better-than-random curriculum when it comes to reducing number of feedbacks on the target task alone but are not better-than-random in desigining a curriculum that decreases the overall number of feedbacks.	Language and Policy Learning from Human-delivered Feedback, Learning behaviors via human-delivered discrete feedback
Proximal Policy Optimization Algorithms	Not directly related to Curriculum Learning but related to Reinforcement Learning.
A Deep Hierarchical Approach to Lifelong Learning in Minecraft	Haven't read it yet, not directly connected to CL but should still be helpful