[Comp-neuro] Review announcement

Ross Gayler r.gayler at mbox.com.au
Tue Jul 29 00:29:59 CEST 2008


I had avoided weighing into this discussion on the grounds that my interest
lies in abstract connectionist models that are quite remote from the
physiological detail that has been the focus here.  However, I would like to
reinforce a comment that Jim Bower made.

> the more sophisticated a coding system, the harder it is likely to be to
distinguish signal from "noise". 

Vector Symbolic Architectures are a family of abstract connectionist models
that use very high-dimensional vectors to represent complex data structures
that might be required for cognition. (Here's a quick challenge question for
you. How might the brain represent "The astronomer believes that the little
star orbits the big star"?)  In VSA these data structures are represented by
the pattern of activity over, say, 10,000 neurons (abstracted as a 10,000
dimensional vector).  This is a thoroughly distributed representation in
which there is no necessary significance to the activity of any individual
neuron - it is only the overall pattern of activity that represents.  

Furthermore, and this is the point I wanted to make, individual patterns of
activity are generally indistinguishable from random vectors.  It is only
the relationship between representing vectors that carries useful
information.  An external observer cannot decode the representing vectors
without knowing what other vectors are stored in the clean-up memory of the
system.  I take these representations to be a very clear demonstration of
Jim's point that in a sophisticated coding system it is harder to
distinguish signal from noise.

This paper (http://cogprints.org/3983/) provides some pointers into the VSA
literature.  The most extensive treatment of a specific instantiation of VSA
is Tony Plate's book
(http://csli-publications.stanford.edu/site/1575864304.html)


Ross Gayler




-----Original Message-----
From: comp-neuro-bounces at neuroinf.org
[mailto:comp-neuro-bounces at neuroinf.org] On Behalf Of jim bower
Sent: Saturday, 26 July 2008 7:31 AM
To: rod rinkus; comp-neuro-bounces at neuroinf.org; minai_ali at yahoo.com
Cc: comp-neuro at neuroinf.org
Subject: [Possible Spam] Re: [Comp-neuro] Review announcement

One last plea for caution. No matter how convenient "noise" may be for
abstract models, one can argue I think pretty convincingly that life itself
differentiates itself from a-biological chemical evolution through its use
of extreme degrees of structure and context specific organization to "beat"
thermodynamics. 

As such anything that smells gaussian, disordered, something you can average
over, or something that is completely generic rather than highly specific to
the circumstances at hand (behavioral or computational) should raise
concern. 

The long history of biology is that the closer you look, the more structure,
not less structure you see.  And everytime I have heard it suggested
otherwise, closer inspection has suggested that we simply weren't giving
biology enough credit. 

One quick example, a number of years ago, in fact during the first summer of
the first computational neuroscience course in woods hole, a well know
investigator of aplesia neurons gave a general MBL talk in which he sought
to identify every conductance in the neuron of interest. In addition to the
big "important" ones he found several small ones, whose presence he chalked
up to sloppyness in protein transcription. 

Given my predispositions, I suggested that he change the temperature of the
bath as aplesia live in tide pools. The next summer he gave a talk on the
remarkable resiliance of neuronal function under different ambient
temperatures confered by the complex conductances present in the cell
membrane.  

While convenient, many of these "noise" mechanisms strike me as being simply
not sophisticated enough. And I will point out now for the last time, that
the more sophisticated a coding system, the harder it is likely to be to
distinguish signal from "noise". 

Jim Bower
Sent via BlackBerry by AT&T

-----Original Message-----
From: "rod rinkus" <rinkus at comcast.net>

Date: Thu, 24 Jul 2008 00:33:59
To: <minai_ali at yahoo.com>
Cc: <comp-neuro at neuroinf.org>
Subject: RE: [Comp-neuro] Review announcement


_______________________________________________
Comp-neuro mailing list
Comp-neuro at neuroinf.org
http://www.neuroinf.org/mailman/listinfo/comp-neuro




More information about the Comp-neuro mailing list