[ExI] Critical take on The Age of Em

[Robin D Hanson]

On Jun 23, 2016, at 1:04 AM, Rafal Smigrodzki <rafal.smigrodzki at gmail.com<mailto:rafal.smigrodzki at gmail.com>> wrote:
Imagine someone gave you dozens of examples of financial spaghetti code from decades ago, all of which do a similar range of financial tasks. And all you have is the object code, not the source code. It would be very hard to abstract from those examples a “generic” financial system capable of doing those financial tasks. That isn’t a remotely easy task. To create a generic brain you have to abstract usefully from the spaghetti object code that is the human brain.

### Ems will be software but not much like today's software. Reasoning by analogy is quite tricky and very uncertain if the analogies break down even mildly.

Today's software has a bewildering variety of languages, approaches, structures. Ems will be minor variations on a theme.

Object code is not obviously modular. Human brains are modular.

Software comes with some programmer remarks in source code. Humans come with whole scientific disciplines devoted to producing descriptions of the human brain on many levels. By assumption these disciplines will have developed tremendously beyond today's level in the scenario we analyze.

Existing software you refer to (financial analysis software from decades ago) was explicitly coded, in great detail and does not self-organize. Human brains are designed to self-organize from basic principles acting on generic hardware and local input data, producing individual detailed structure (memories). As we discussed here before, the size and complexity of the generic principles in human brain is orders of magnitude smaller than the size of data structures produced by self-organization. This implies that you need to know only a small number of principles to build a self-organizing device, the generic human mind em.

Yes many kinds of software today don’t do much learning, but we do have many real systems that do learn, usually based on statistics. (We also know of many software systems where the object code is much larger than the source code for other reasons.) So if you think that makes a huge difference, then imagine instead that someone gave you dozens of examples of statistical analysis code from decades ago. You’d have the same problem.

You don’t know that human brains are any more modular than is typical software. You don’t know that it only embodies a small number of principles, without masses of other implementation details also required for it work. And you don’t know that the many different parts of the brain are all written in the same “language”.

I guess you don’t believe in software rot? You think it possible to teach real legacy software systems and make them young again? People try to do this with refactoring, but it is very hard and has only limited success.

### Think about the following: A brain-machine interface uses a few hundred electrodes and signal processing software to bypass large chunks of signal processing wetware and to replicate limb movement. Using the crudest equipment you bypass the cerebellum, some subcortical structures, the spinal cord and you successfully move a limb. This means you can get useful functional division of the human brain even without detailed cell-by-cell access. And ems will have random access to all cell states, and a huge amount of knowledge of their interactions. We already know that the human brain is highly modular, and there are many forms of memory and learning implemented in distinct modules. The em will be able to isolate and keep some protected content (most important personal memories and attitudes) and flush away cruft, something not easy to achieve with the spaghetti software you mention, because the em is a different type of software.

This one datum doesn’t indicate that the brain is any more modular that most software systems. Saying it is a “different type” and thus vastly easier to understand and modify just sounds like wishful thinking.

Robin Hanson rhanson at gmu.edu<mailto:rhanson at gmu.edu>
Future of Humanity Inst., Oxford University
Assoc. Prof. Economics, George Mason University
See my new book: http://ageofem.com









-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20160623/3505a613/attachment.html>