The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learning release_uf4nviubnffz3f2qqui24zc3ei

by Yujin Tang, David Ha

Released as a article .

2021  

Abstract

In complex systems, we often observe complex global behavior emerge from a collection of agents interacting with each other in their environment, with each individual agent acting only on locally available information, without knowing the full picture. Such systems have inspired development of artificial intelligence algorithms in areas such as swarm optimization and cellular automata. Motivated by the emergence of collective behavior from complex cellular systems, we build systems that feed each sensory input from the environment into distinct, but identical neural networks, each with no fixed relationship with one another. We show that these sensory networks can be trained to integrate information received locally, and through communication via an attention mechanism, can collectively produce a globally coherent policy. Moreover, the system can still perform its task even if the ordering of its inputs is randomly permuted several times during an episode. These permutation invariant systems also display useful robustness and generalization properties that are broadly applicable. Interactive demo and videos of our results: https://attentionneuron.github.io/
In text/plain format

Archived Files and Locations

application/pdf  4.0 MB
file_pi7gqgsxpvdjfjsmhegng2r6ya
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2021-09-29
Version   v2
Language   en ?
arXiv  2109.02869v2
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: d9649a31-9979-492d-be80-e11e4fc2f718
API URL: JSON