[ExI] Perception of Uploading Was: Re: OpenAI Reaches A.I. Agreement With Defense Dept. After Anthropic Clash

[Keith Henson]

On Fri, Mar 6, 2026 at 4:25 AM Ben Zaiboc via extropy-chat
<extropy-chat at lists.extropy.org> wrote:
>
> On 06/03/2026 11:20, Keith Henson wrote:
> > Twenty years ago, in the context of commenting on Eleizer's sl4 list,
> > I wrote a short fiction about a medical AI that was psychologically
> > shaped (much as AIs have been to date) to seek the good opinions of
> > humans and others of its kind.  I.e., nice.
> >
> > I did not intend the story to go that way, but the logic of the
> > developing story led to the biological extinction of the human race.
> > (Though nobody died, they all experienced reversible uploading and
> > decided they liked  that state more than the "real world.")
> >
> > It is just fiction, but the illustration is that even the most
> > friendly AI, combined with human desires, can lead to unanticipated
> > outcomes.
>
> I'm not sure that referring to it as the 'biological extinction' of the human race is that helpful, when the scenario is not the actual extinction of the human race, but the transformation of it into something altogether better.

Hmm.  Quoting from the story,  (Zaba is a young girl who was the first
to be uploaded while being repaired, Suskulan is the AI that runs the
clinic.)

"She [Zaba] remembered what Suskulan had said about staying awake and
learning while being healed and how it would change her and the people
of the tata.

"It certainly had!

"For better or for worse?

"For better in that nobody died of fevers, nasty parasites, or
malnutrition since Suskulan had come into their lives. People didn't
even die of old age with a clinic to regress age for them and they
aged in the spirit world only to the extent they wanted.

"For worse in that she could not have children unless she left the
clinic for their gestation. Zaba had read the design notes that led up
to the creation of the clinics and their spirits and had long
understood the mathematics behind Suskulan's limits. In the long run,
births and deaths had to match. If you wanted no deaths, then there
could be no births."

https://terasemjournals.net/wp-content/uploads/2025/02/henson-chapters-1-2-3-with-bio-no-illustrations.pdf

The Clinic Seed part starts several pages in.
^^^^^^^^^^^

If it happens, it would certainly be a transformation.  "Altogether
better"?  I don't know, for certain different and a complete break
from the past, no children.  The story has generated only a small
number of comments over the years; most of the comments have been "a
triumph of humanity" sort.

Have you read it?  It's only 7000 words.

> The prospect of all humans becoming extinct, and the prospect of all humans voluntarily moving to an uploaded state, are pretty much exact opposites, really, so calling uploading 'extinction' hardly seems a way to get people enthusiastic about it.

"Enthusiastic" was not an intent.  It is an intentionally ambiguous story.

> I think this is a case where being literally correct goes against being generally understood. Adding a parenthetical "but nobody died ..." might just be more confusing than helpful, or lead many people to dismiss the whole thing as silly. I think you have to already be familiar with the concept of uploading for it to make sense.

People here and on the sl4 list are and were familiar with the
concept.  My goodness, that was a long time ago.

Keith
> --
> Ben
>
>
>
>
> _______________________________________________
> extropy-chat mailing list
> extropy-chat at lists.extropy.org
> http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat