Title: Wormnet Reinforcement Learning without Global Back-Propagation
Language: English
Authors: Lemmel, Julian 
Keywords: RL; Backpropagation
Advisor: Grosu, Radu 
Issue Date: 2020
Number of Pages: 42
Qualification level: Diploma
Reinforcement Learning (RL) aims at creating controllers for discrete and continuous problems and was initially inspired by neuroscience. However, the most successful methods are relying on backpropagation for calculating the gradients of the loss-function. The backpropagation algorithm is considered to be biologically implausible suggesting that it will not suffice when striving for human-like learning abilities. Neuroscience has brought forth different models of synaptic plasticity by observing isolated neurons. Such models could serve as alternatives to the ubiquitous backpropagation algorithm for calculating changes to network parameters. Neural Regulatory Networks are special RNNs whose inner states are calculated according to dynamics derived from biological observations. In this thesis, a novel framework based on state-of-the-art RL techniques and using NRNs, is introduced and experimented with by applying it to a cartpole balancing task. Two different methods of incorporating learning rules based on models of synaptic plasticity are investigated: the custom gradients method replaces the real gradient calculated by backpropagation with a biologically plausible synaptic plasticity rule, the plasticity dynamics method leaves the gradients unchanged but introduces additional plasticity dynamics that act throughout the entire unrolling of network states. Both methods were tested with three different learning rules: hebb’s rule, oja’s rule and the BCM rule. The results suggest that training can be accelerated when using the BCM rule.
URI: https://doi.org/10.34726/hss.2020.81325
DOI: 10.34726/hss.2020.81325
Library ID: AC15760934
Organisation: E191 - Institut für Computer Engineering 
Publication Type: Thesis
Appears in Collections:Hochschulschrift | Thesis

Files in this item:

Show full item record

Google ScholarTM


Items in reposiTUm are protected by copyright, with all rights reserved, unless otherwise indicated.