[ExI] AGI will likely be uncontrollable and very strange

[Ben Zaiboc]

On 24/10/2026 06:08, Keith Hensons wrote:
> On Fri, Oct 23, 2026 at 11:13 AM Stuart LaForge <avant at sollegro.com> wrote:
>
>     On 2026-10-23 08:45, Keith Henson wrote:
>
> > I made a case (in a fictional context) 20 years ago that the problem is humans, not AIs. Even the best-aligned AI can result in the biological extinction of humans.
>
> > "The tata (village) was home to the leopard for the rest of his life.)
>
> "The Clinic Seed" shows a possibility, but I think there would almost certainly be holdouts that decide to maintain physical existence warts and all and eschew virtual life for duty, preference, or on principle no matter how tempting the Matrix might become. There is no reason why AI as advanced as the clinic seed AI could not, through genetic engineering and selective breeding, create biological or cybernetic transhumans, that don't suffer from old age or other diseases and afflictions.
>     Stuart LaForge
>
>
> The other connected chapters included people who stayed out of the uploaded state, partly because a story needs characters and uploaded characters are really hard to write about. But most, 99% did enter the uploaded state which left the physical world depopulated. I obviously don't know if this is realistic, it's just a story.
>
> The real future may be stranger than any of us can imagine.


It probably will be. The most probable outcome that I can imagine, at least for now, is that the physical world will not be depopulated exactly, but populated by more than just unmodified, 'original' biological humans, who may be a small minority amongst a large variety of physical forms, from what we would regard as 'robots', to cyborgs, to what would look like human and other biological beings to us, that are actually far from that, as well as other forms that we probably can't (or don't) imagine just yet. Some of these may well have a dual existence, in both the virtual and physical worlds.

I can't honestly see the world reverting to a wilderness in any event, even if all the thinking happens in VR. Physical resources are physical resources, and won't be wasted like that. Parks and zoos, reservations, sure, but most of the world will be useful, if only for raw materials for building physical infrastructure. See 'After Life' by Simon Funk (well worth reading anyway).
-- 
Ben
"The world will not be destroyed by those who do evil, but by those who watch them without doing anything." - Pablo Casals, via Albert Einstein