[ExI] Why “Everyone Dies” Gets AGI All Wrong by Ben Goertzel

[Keith Henson]

On Sun, Oct 5, 2025 at 3:55 AM Ben Zaiboc via extropy-chat
<extropy-chat at lists.extropy.org> wrote:
>
> On 05/10/2025 04:17, Keith Henson wrote:
>
> Full molecular nanotechnology that is up to mapping out the brain
> should be able to implant memory.  I don't say it will be easy, but
> with only modest life extension, I can wait.  One way uploading (which
> Hans Moravec proposed) seems like buying a car without a test drive.
>
> Well, I'd rather wait too, and hopefully, with luck and care, will be able to, but some people don't have that luxury. Cryonics may be a solution, but it may be a one-way trip as well.
>
> Perhaps implanting memory into a biological brain will be a matter of tweaking synaptic weighting, which should be doable given non-destructive scanning technology, but it probably will also involve changing neural connections, which is rather different. Not impossible, granted, but the main thing that occurs to me is that an uploaded mind could rapidly develop beyond the ability of any biological brain to contain.

That was what Suskulan warned Zaba about when she left the clinic.  It
may or may not be a problem

> For example, if I was uploaded today, one of the first things I'd want to change (after taking a subjective year or so to get acclimatised) would be to expand my working memory and to improve my ability to make sense of maths and statistics. Given that these current limitations are probably a result of my brain structure, we'd be talking about major refurbishment to transfer those changes back to my biological brain. Then there's things like extra sensory modalities, improved visual imagery, and all the other things that would be doable in an uploaded mind, but not in biology.
> I imagine that being squeezed back into your original brain would be more like a lobotomy than anything else.

Perhaps.  I see the memory updating process as being continuous.  You
would lose things like the ability to run fast by moving back into
your brain.

> Constructing a new brain from scratch seems to be the best solution, and as I said, that would probably not be biological anyway. Given uploading tech., and the level of technological acceleration that implies, biology will probably be last-year's tech. by then.
>
>
> The chapter following the Clinic Seed goes into an urban setting where
> the inactive bodies are stored under the buildings they lived in.
>
> Ok, but why do that when the technology exists to create bodies on demand? The only reasons I can think of are psychological, and with the probable time-difference between virtual and meat-space, people would probably get beyond that within minutes or seconds. It would be like keeping an old and decrepit empty house that you only briefly visit every decade or so, that no-one else can use, when there are much better hotels available. Even the fittest 17-year-old bio body is going to seem clunky, clumsy, stiff and exhausting after 10 minutes as an upload. And a 40-year-old body? Forget it, there are much better ways for masochists to get their jollies.
>
> The thing that interests me here is not so much the end-result but the path to it. Medical interventions as a front-end would be a good strategy, but I can't see the story of Zaba?, was it? being typical. And I can't see any government letting an AI system as powerful as that have free reign to do whatever people want it to do, re. medical things and uploading. I reckon a messy interim period is inevitable, and something like Neil Asher's 'quiet war' would be the very best we can hope for, where the AIs take over without much turmoil and death. Probably unlikely, though.

I see governments as irrelevant.  They are made of people who just
cannot react fast enough.  There was a worm that doubled every 8.5
seconds.  It infected every system on the net before humans could
react.  Even the pace of AI development is too fast for governments to
keep up.

> The important question might well be: "What can we do to prepare to survive the interim period (assuming anyone can), in the next 6 months to 6 years?" Beyond 6 years, I reckon it's completely pointless to speculate.

I think it is safe to say the demand for "compute" will increase and
eventually move into space.

Keith

> Adrian was right to say the singularity won't be tomorrow. It might be the day after, though.
>
> --
> Ben
>
> _______________________________________________
> extropy-chat mailing list
> extropy-chat at lists.extropy.org
> http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat