[ExI] Why “Everyone Dies” Gets AGI All Wrong by Ben Goertzel

[Ben Zaiboc]

On 05/10/2025 04:17, Keith Henson wrote:
> Full molecular nanotechnology that is up to mapping out the brain
> should be able to implant memory.  I don't say it will be easy, but
> with only modest life extension, I can wait.  One way uploading (which
> Hans Moravec proposed) seems like buying a car without a test drive.

Well, I'd rather wait too, and hopefully, with luck and care, will be 
able to, but some people don't have that luxury. Cryonics may be a 
solution, but it may be a one-way trip as well.

Perhaps implanting memory into a biological brain will be a matter of 
tweaking synaptic weighting, which should be doable given 
non-destructive scanning technology, but it probably will also involve 
changing neural connections, which is rather different. Not impossible, 
granted, but the main thing that occurs to me is that an uploaded mind 
could rapidly develop beyond the ability of any biological brain to 
contain. For example, if I was uploaded today, one of the first things 
I'd want to change (after taking a subjective year or so to get 
acclimatised) would be to expand my working memory and to improve my 
ability to make sense of maths and statistics. Given that these current 
limitations are probably a result of my brain structure, we'd be talking 
about major refurbishment to transfer those changes back to my 
biological brain. Then there's things like extra sensory modalities, 
improved visual imagery, and all the other things that would be doable 
in an uploaded mind, but not in biology.
I imagine that being squeezed back into your original brain would be 
more like a lobotomy than anything else.

Constructing a new brain from scratch seems to be the best solution, and 
as I said, that would probably not be biological anyway. Given uploading 
tech., and the level of technological acceleration that implies, biology 
will probably be last-year's tech. by then.


> The chapter following the Clinic Seed goes into an urban setting where
> the inactive bodies are stored under the buildings they lived in.

Ok, but why do that when the technology exists to create bodies on 
demand? The only reasons I can think of are psychological, and with the 
probable time-difference between virtual and meat-space, people would 
probably get beyond that within minutes or seconds. It would be like 
keeping an old and decrepit empty house that you only briefly visit 
every decade or so, that no-one else can use, when there are much better 
hotels available. Even the fittest 17-year-old bio body is going to seem 
clunky, clumsy, stiff and exhausting after 10 minutes as an upload. And 
a 40-year-old body? Forget it, there are much better ways for masochists 
to get their jollies.

The thing that interests me here is not so much the end-result but the 
path to it. Medical interventions as a front-end would be a good 
strategy, but I can't see the story of Zaba?, was it? being typical. And 
I can't see any government letting an AI system as powerful as that have 
free reign to do whatever people want it to do, re. medical things and 
uploading. I reckon a messy interim period is inevitable, and something 
like Neil Asher's 'quiet war' would be the very best we can hope for, 
where the AIs take over without much turmoil and death. Probably 
unlikely, though.

The important question might well be: "What can we do to prepare to 
survive the interim period (assuming anyone can), in the next 6 months 
to 6 years?" Beyond 6 years, I reckon it's completely pointless to 
speculate.

Adrian was right to say the singularity won't be tomorrow. It might be 
the day after, though.

-- 
Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20251005/81537c62/attachment.htm>