[Comp-neuro] role of noise in learning

Chi-Sang Poon cpoon at MIT.EDU
Thu Jul 24 15:53:03 CEST 2008


Dear Wolfgang and All,

    Here is another reference for noise-driven reinforcement learning in the
context of adaptive control:

IEEE Trans Syst Man Cybern B Cybern. 2001;31(2):173-86.Links
    A Hebbian feedback covariance learning paradigm for self-tuning optimal
control.
    Young DL, Poon CS.
    We propose a novel adaptive optimal control paradigm inspired by Hebbian
covariance synaptic adaptation, a preeminent model of learning and memory as
well as other malleable functions in the brain. The adaptation is driven by
the spontaneous fluctuations in the system input and output, the covariance
of which provides useful information about the changes in the system
behavior. The control structure represents a novel form of associative
reinforcement learning in which the reinforcement signal is implicitly given
by the covariance of the input-output (I/O) signals. Theoretical foundations
for the paradigm are derived using Lyapunov theory and are verified by means
of computer simulations. The learning algorithm is applicable to a general
class of nonlinear adaptive control problems. This on-line direct adaptive
control method benefits from a computationally straightforward design, proof
of convergence, no need for complete system identification, robustness to
noise and uncertainties, and the ability to optimize a general performance
criterion in terms of system states and control signals. These attractive
properties of Hebbian feedback covariance learning control lend themselves
to future investigations into the computational functions of synaptic
plasticity in biological neurons.
PMID: 18244780 [PubMed - in process]



-----Original Message-----
From: comp-neuro-bounces at neuroinf.org
[mailto:comp-neuro-bounces at neuroinf.org] On Behalf Of Wolfgang Maass
Sent: Thursday, July 24, 2008 8:02 AM
To: comp-neuro at neuroinf.org
Cc: nelson at brandeis.edu
Subject: [Comp-neuro] role of noise in learning

I would like to add to your discussion that "noise" is obviously
needed for reward-based learning in networks of neurons:

If such networks have to learn without a supervisor (which tells the 
neurons during training when they should fire), they have to explore 
different ways of responding to a stimulus, until the come across 
responses that are "rewarded" because they provide good system 
performance. This exploration would appear as "noise" in most analyses. 
In fact, one might conjecture that networks of neurons are genetically 
endowed with the capability to go through rather clever exploration 
patterns (i.e, particular types of "noise"), in order to enable fast 
convergence of such reinforcement learning schemes.

The role of noise in reward-based learning has been analyzed by a number 
of people, see #183 on
http://www.igi.tugraz.at/maass/publications.html
for a very recent contribution (and references to earlier work).

-Wolfgang

-- 
Prof. Dr. Wolfgang Maass
Institut fuer Grundlagen der Informationsverarbeitung
Technische Universitaet Graz
Inffeldgasse 16b ,   A-8010 Graz,  Austria
Tel.:  ++43/316/873-5811
Fax   ++43/316/873-5805
http://www.igi.tugraz.at/maass/Welcome.html
_______________________________________________
Comp-neuro mailing list
Comp-neuro at neuroinf.org
http://www.neuroinf.org/mailman/listinfo/comp-neuro



More information about the Comp-neuro mailing list