[ExI] The second step towards immortality

[Samantha Atkins]

On 01/04/2014 05:01 AM, Anders Sandberg wrote:
>
> Copied unprotected minds are potentially enslaved minds. If I get a 
> copy of your mind I can torture it for fun forever. I can try out all 
> sorts of manipulations to get it to do what I want - whether running a 
> genetic algorithm trying out arguments on copies or straight editing 
> of pain/reward pathways. And then I could put copies of you to work on 
> whatever I like - like doing your job, maybe outcompeting the 
> original. Or just use copies for my psychological research. There are 
> loads of very nasty things I could do to a copied mind if I had no 
> ethics.
>
> The smaller minds are compared to the typical computational 
> infrastructure the easier it is to hide an airgapped slave pen 
> computer in my cellar.

Well sure.  But I know you are a much nicer person than that.  And you 
know that if you were ever caught no copy of any decent intelligence 
will likely give any copy of yourself any runtime. So not a big problem.

Besides, I doubt that a hacked human brain would be as efficient as a 
purpose built AI/AGI for the task.  This is likely relevant because the 
tech to copy and hack brains would likely make AGIs and special purpose 
AIs relatively trivial.   These would be wildly popular as they would 
have no ethical and possibly prosecutable criminal overhead.

- samantha