[ExI] Why “Everyone Dies” Gets AGI All Wrong by Ben Goertzel

[Ben Zaiboc]

On 04/10/2025 20:46, Keith Henson wrote:
> Ben, I don't think you have thought it through.

Almost certainly not! Thinking it through is what we're doing here, in 
this distributed brain we call Exi-Chat.
> Technology to upload should permit memory updating in a biological
> brain.

I don't see that uploading necessarily implies this. It seems a much 
harder problem, and might not be possible at all without redesigning the 
brain, then rebuilding it, at which point we're probably not talking 
about biology any more, making it pretty pointless.

Besides which, there very well might not be any brain left to update, 
after uploading. I suspect that destructive scanning will always be 
easier and quicker than non-destructive scanning, and developed sooner.

A synthetic body and brain, sure. In fact I think it could be a standard 
feature of uploading. Instead of uploading to a server somewhere, 
together with millions of other minds, you could be uploaded to your own 
private server, residing in a synthetic body, with the ability to 
communicate with millions of other minds, build shared virtual spaces, 
etc., as well as act in the physical world.

Or even both, with a mind in a shared server, and another mind in a 
synth. body, updating each other. Or more than two. Mental RAIDs!

(By 'synthetic', I don't mean metal and plastic etc., I mean a 
technology that we don't yet have, that goes beyond biology. Some form 
of nanotech., almost certainly)

> Which means you can upload for a weekend and resume being a
> meat human on Monday.  The only limit that might be imposed is a "one
> at a time" rule for practical physical reasons
>
> If AI clinics like the one in The Clinic Seed become common, I don't
> see how religious or authoritarian regimes can do anything about
> people uploading.

Only if the authoritarian regimes (which includes religious ones) lose 
their power over their subjects, which would mean they don't exist any more.
And don't forget that one form of power over people is the ability to 
persuade (as opposed to coerce) them that they do or don't want to do 
particular things.
Certain current regimes seem to be pretty good at that.

The 'Clinic Seed' style of uploading availability is only one 
possibility (and to be honest, probably not the most likely). There are 
many other models that have been thought of, and more that haven't. 
Different models would probably suit different places as well (how would 
you adapt the Clinic Seed concept for use in an urban setting in, say, 
communist China? London? Nairobi?).

But there's probably not much point speculating, because we're talking 
about singularity territory. Fun as it might be, any speculation is 
almost certainly bound to be wrong.


> On Sat, Oct 4, 2025 at 3:23 AM Ben Zaiboc via extropy-chat
> <extropy-chat at lists.extropy.org> wrote:
>> On 04/10/2025 04:00, Keith Henson wrote:
>>> Uploaded humans living in private spaces don't have to agree on
>>> anything.  Their simulated world can be anything they like, including
>>> simulated slaves to beat.  Not my ideal world, but I am sure there
>>> will be some who want it.
>> Uploading only solves the problem for people who want to and can upload
>> (and have control of their simulations).
>>
>> I expect it would be very likely that there will be plenty of people who
>> want to subjugate 'real' people, in the 'real' world, and would reject
>> uploading, as well as plenty of people who are prevented from uploading
>> (like everyone in all the autocratic and theocratic regimes for a start,
>> so several billion).
>>
>> Some kind of consensus on the 'correct' (or even acceptable) values is
>> extremely unlikely.
>>
>> The 'Universal declaration of human rights' is a lot less universal than
>> you might think, for example. Article 2* in particular is simply false,
>> if you take it as stating a fact rather than expressing an aspiration.
>> Certainly many, if not most, nations disagree with it in practice. And
>> apart from anything else, 'Freedom to upload' isn't in it, of course.
>> I'm now wondering if even some western nations might not enact
>> legislation to outlaw uploading, once it becomes possible. It wouldn't
>> surprise me, there are plenty of people who regard it as tantamount to
>> suicide, even among people who regard themselves as transhumanists or
>> materialists. It wouldn't be hard for a government to conclude that it
>> should be illegal (or that uploads don't count as human, and don't have
>> any rights** at all). Look at how most governments responded to
>> cryptocurrencies. If anything threatens their control, they will oppose
>> it. This is just as true of liberal democracies as it is of all the
>> other forms of government, the main advantage of democracies is that
>> they change quicker and with less disruption than more traditional
>> dictatorships.
>>
>> Voluntary agreement on a single set of values to cover all humans is not
>> just unlikely, it's probably impossible (just another reason why
>> Yudkowsky's book is wrong).
>>
>> --
>> Ben
>>
>> * "Everyone is entitled to all the rights and freedoms set forth in this
>>    Declaration, without distinction of any kind, such as race, colour,
>> sex, language, religion, political or other opinion, national or social
>> origin, property, birth or other status. Furthermore, no distinction
>> shall be made on the basis of the political, jurisdictional or
>> international status of the country or territory to which a person
>> belongs, whether it be independent, trust, non-self-governing or under
>> any other limitation of sovereignty"
>>
>>
>> ** I don't remember who it was that pointed out that your rights only extend as far as your ability to enforce them.
>>
>> Ben

-- 
Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20251004/acef08e8/attachment-0001.htm>