Robot Deep Reinforcement Learning: Tensor State-Action Spaces and Auxiliary Task Learning with Multiple State Representations release_3hfgb2n32fglhfqqzlh7gae7ee

by Devin Schwab

Published by Carnegie Mellon University.

2020  

Abstract

A long standing goal of robotics research is to create algorithms that can automatically learn complex control strategies from scratch. Part of the challenge ofapplying such algorithms to robots is the choice of representation. Reinforcement Learning (RL) algorithms have been successfully applied to many different robotictasks such as the Ball-in-a-Cup task with a robot arm and various RoboCup robot soccer inspired domains. However, RL algorithms still suffer from issues of large training time and large amounts of required training data. Choosing appropriate representations for the state space, action space and policy can go a long way towards reducing the required training time and required training data.This thesis focuses on robot deep reinforcement learning. Specifically, how choices of representation for state spaces, action spaces, and policies can reduce training time and sample complexity for robot learning tasks. In particular the focus is on two main areas:1. Transferrable Representations via Tensor State-Action Spaces2. Auxiliary Task Learning with Multiple State RepresentationsThe first area explores methods for improving transfer of robot policies across environment changes. Learning a policy can be expensive, but if the policy can betransferred and reused across similar environments, the training costs can be amortized. Transfer learning is a well-studied area with multiple techniques. In this thesis we focus on designing a representation that makes for easy transfer. Our method maps state-spaces and action spaces to multi-dimensional tensors designed to remain a fixed dimension as the number of robots and other objects in an environmentvaries. We also present the Fully Convolutional Q-Network (FCQN) policy representation, a specialized network architecture that combined with the tensor representationallows for zero-shot transfer across environment sizes. We demonstrate such an approach on simulated single and multi-agent tasks inspired by RoboCup Small Size League (SSL) and a modified version of At [...]
In text/plain format

Archived Files and Locations

application/pdf  13.5 MB
file_s5g5byfh4bborjoqugg3xhmxoe
s3-eu-west-1.amazonaws.com (publisher)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article-journal
Stage   published
Date   2020-10-16
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: 53d8b3ec-f48e-4b80-bdf3-87a4e5080560
API URL: JSON