[ExI] thawing of ai winter?

[Rafal Smigrodzki]

On Sat, Apr 18, 2020 at 7:17 AM Kunvar Thaman <
f20170964 at pilani.bits-pilani.ac.in> wrote:

> The convergence of GANs is still not proved, correct?
>

### Explain?


> >As the brain matures in contact with the real world, each of the
> pre-wired layers is then fleshed out, and thanks to the pre-wired scaffold
> the amount of data needed to optimize each layer is low.
>
> We don't yet know distributed information is coded in brains, but we do
> know that the cortex is not so specialized as to be able to represent only
> one modality in one region. For example, consider experiments of re-wiring
> to enable seeing with tongue, etc. So, is it pre-wired, really?
>

### Yes, the cortex has a relatively uniform general wiring pattern
throughout, and many cortical areas can take on tasks that are different
from their usual ones. However, you need to consider the larger picture, of
which the cortex is but one part.

Our nervous system is much more than the cortex, most of it is hardwired or
partially hardwired. The pattern of nuclei and tracts within the spinal
cord, brainstem and the peripheral nervous system is coded in the genome,
with automated routines achieving connection milestones well before birth.
This is why babies in the womb kick - they are calibrating the reciprocal
reflexive connections within the spinal cord. The pattern of connections in
the basal ganglia is very complex and innate. The cortex cannot work at all
without a huge amount of input from the basal ganglia - if your caudate or
thalamus are destroyed, you become severely demented, even if the cortex is
untouched.

And of course, the ability of one cortical area to learn different tasks
does not mean that all cortical areas are computationally equivalent. While
the general pattern of short intracortical connections is as I said
relatively uniform, there is variability, not as much as the differences
between various subcortical structures but still substantial. If you look
at the cortex under the microscope, which is a very crude tool, you can
already discern a lot of subtle variability (Broadman's cortical areas),
and the closer you look, using various functional techniques, the more
complex it is. The wiring pattern for the macaque visual cortex has more
than a hundred distinct functional parts, and it's just a very small part
of the overall structure.  Some of that is likely to be encoded
genetically, with a suite of task-specific connection patterns (routines)
ready to be deployed in a cortical area, and the specific routine depending
on pattern of input from basal ganglia. This accounts for the ability to
re-wire sensory areas to respond to different formats of sensory input
(visual, tactile, etc.) and it is still pre-wired, in the genome.

It is estimated that the brain contains more than 10,000 distinct types of
neurons and hundreds of thousands of distinct neuroanatomical areas. But
this is not all - the pattern of connections between cortical areas is also
very non-random. There are hundreds of thousands of tracts connecting
different areas of the cortex, and many of these connections are under
close genetic (i.e. pre-wired) control. For example, disruption of the
Foxp2 gene causes a language development impairment due to mis-wired
connections to and from the Broca's area. Doubtless there are thousands of
other specific genetically determined connections within the cortex.

This clearly shows that the cortex does not just wire itself up from
scratch, like deep-learning networks did when research on them started
years ago. The cortex is built atop a huge hardwired pile of complex
computing machinery, it is dependent on the pre-processed input from that
machinery and it relies on a lot of hardwired machinery to process its
outputs. Intracortical connections are also pre-wired and genetically
controlled, not relying on a uniform simple algorithm for creating
connections.

------------------------------------------

> What do you mean pre-wired? Our brains are constantly changing, in fact,
> every bye of information you process causes a physical change in the brain
> structure.
>
>
### As I was explaining above, by "pre-wired" I mean the complex pattern of
nuclei and tracts throughout the spinal cord, brainstem, cerebellum, basal
ganglia that is largely or even exclusively genetically determined, as well
as the more responsive to experience but still genetically planned pattern
of connections within the cortex. The process of individual learning
changes very little on the general wiring pattern, instead works on the
microscale of individual synapses and cortical columns. This is a bit
analogous to hardware and software in a computer. The general inherited
(genetic) wiring pattern is the hardware, and individual synapse creation
in response to various inputs is analogous to software.

In this analogy, the contest between modern deep learning systems and
biological brains is like a contest between a general software emulation
competing against a hardware-accelerated machine highly optimized for a
complex task. You need stupendous general computing power to develop an
emulator capable of outrunning a task-oriented hardware accelerated machine.

Rafal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20200419/6382bc10/attachment.htm>