[ExI] Critical take on The Age of Em

[Rafal Smigrodzki]

On Wed, Jun 22, 2016 at 9:03 AM, Robin D Hanson <rhanson at gmu.edu> wrote:

> On Jun 21, 2016, at 9:36 PM, Rafal Smigrodzki <rafal.smigrodzki at gmail.com>
> wrote:
>


> Yet when 100% of experts on AI say em is not likely to happen before AI,
> you do not take their opinion at face value. Something does not fit here.
>
> 100% does not remotely sound correct. I request a citation for that
> figure.
>

### In chapter 4 you quote a survey of AI researchers (Mueller and Bostrom
2014), and you mention that ""none of these 29 thought that brain emulation
"might contribute the most" to human level AI"". I take this to mean that
they think human level AI won't be first implemented as a em. If the first
AI was in fact an em, then ems would be contributing the most (i.e. all) to
first human level AI. If ems are not contributing the most to human level
AI, then the first human level AI is not an em.

--------------------

>  Also you aren’t using the phrase “intelligence explosion” the way it is
> usually used. But the usual definition, these observed bursts of progress
> don’t count.
>

### Well, yes, I called it local intelligence explosion but I think that
the usage does point to some interesting parallels between I.J. Good's
intelligence explosion and the AlphaGo. I'll write more on that in a
separate post.

--------------------
Robin wrote:

Imagine someone gave you dozens of examples of financial spaghetti code
> from decades ago, all of which do a similar range of financial tasks. And
> all you have is the object code, not the source code. It would be very hard
> to abstract from those examples a “generic” financial system capable of
> doing those financial tasks. That isn’t a remotely easy task. To create a
> generic brain you have to abstract usefully from the spaghetti object code
> that is the human brain.
>

### Ems will be software but not much like today's software. Reasoning by
analogy is quite tricky and very uncertain if the analogies break down even
mildly.

Today's software has a bewildering variety of languages, approaches,
structures. Ems will be minor variations on a theme.

Object code is not obviously modular. Human brains are modular.

Software comes with some programmer remarks in source code. Humans come
with whole scientific disciplines devoted to producing descriptions of the
human brain on many levels. By assumption these disciplines will have
developed tremendously beyond today's level in the scenario we analyze.

Existing software you refer to (financial analysis software from decades
ago) was explicitly coded, in great detail and does not self-organize.
Human brains are designed to self-organize from basic principles acting on
generic hardware and local input data, producing individual detailed
structure (memories). As we discussed here before, the size and complexity
of the generic principles in human brain is orders of magnitude smaller
than the size of data structures produced by self-organization. This
implies that you need to know only a small number of principles to build a
self-organizing device, the generic human mind em.

-------------
Robin wrote:

I guess you don’t believe in software rot? You think it possible to teach
real legacy software systems and make them young again? People try to do
this with refactoring, but it is very hard and has only limited success.

### Think about the following: A brain-machine interface uses a few hundred
electrodes and signal processing software to bypass large chunks of signal
processing wetware and to replicate limb movement. Using the crudest
equipment you bypass the cerebellum, some subcortical structures, the
spinal cord and you successfully move a limb. This means you can get useful
functional division of the human brain even without detailed cell-by-cell
access. And ems will have random access to all cell states, and a huge
amount of knowledge of their interactions. We already know that the human
brain is highly modular, and there are many forms of memory and learning
implemented in distinct modules. The em will be able to isolate and keep
some protected content (most important personal memories and attitudes) and
flush away cruft, something not easy to achieve with the spaghetti software
you mention, because the em is a different type of software.

Rafał
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20160623/2344867b/attachment.html>