Reinforcement Teaching
release_c2226jnpmfhvvjh25d3jyuseqy
by
Alex Lewandowski, Calarina Muslimani, Dale Schuurmans, Matthew E. Taylor, Jun Luo
2022
Abstract
Meta-learning strives to learn about and improve a student's machine learning
algorithm. However, existing meta-learning methods either only work with
differentiable algorithms or are hand-crafted to improve one specific component
of an algorithm. We develop a unifying meta-learning framework, called
Reinforcement Teaching, to improve the learning process of any algorithm. Under
Reinforcement Teaching, a teaching policy is learned, through reinforcement, to
improve a student's learning. To effectively learn such a teaching policy, we
introduce a parametric-behavior embedder that learns a representation of the
student's learnable parameters from its input/output behavior. Further, we use
learning progress to shape the teacher's reward, allowing it to more quickly
maximize the student's performance. To demonstrate the generality of
Reinforcement Teaching, we conduct experiments where a teacher learns to
significantly improve both reinforcement and supervised learning algorithms,
outperforming hand-crafted heuristics and previously proposed parameter
representations. Results show that Reinforcement Teaching is capable of not
only unifying different meta-learning approaches, but also effectively
leveraging existing tools from reinforcement learning research.
In text/plain
format
Archived Files and Locations
application/pdf 6.1 MB
file_n4ylbcwwcbgjdhslj45lhfgf24
|
arxiv.org (repository) web.archive.org (webarchive) |
2204.11897v2
access all versions, variants, and formats of this works (eg, pre-prints)