[ExI] The second step towards immortality

[Anders Sandberg]

On 2014-01-04 05:48, Martin Sustrik wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 04/01/14 00:35, Anders Sandberg wrote:
>
>> Exactly. Peter Eckersly and me have been looking at computer
>> security for uploaded minds, and it is a worrying problem. If you
>> can be edited or copied you are in deep trouble. If homeomorphic
>> encryption can be made effective enough to keep uploads running,
>> the future looks much brighter.
>
> I would say copying is not much of a problem. Copied mind would work
> in the same way as the original mind. The two can even synchronise via
> a global resource such as Bitcoin blockchain: i.e. if payment is about
> to be made, the program checks whether the transfer already took place
> and if so, it skips the payment.

Copied unprotected minds are potentially enslaved minds. If I get a copy 
of your mind I can torture it for fun forever. I can try out all sorts 
of manipulations to get it to do what I want - whether running a genetic 
algorithm trying out arguments on copies or straight editing of 
pain/reward pathways. And then I could put copies of you to work on 
whatever I like - like doing your job, maybe outcompeting the original. 
Or just use copies for my psychological research. There are loads of 
very nasty things I could do to a copied mind if I had no ethics.

The smaller minds are compared to the typical computational 
infrastructure the easier it is to hide an airgapped slave pen computer 
in my cellar.


-- 
Anders Sandberg
Future of Humanity Institute
Oxford University