[ExI] Fwd: thought experiment part 1

[Colin Hales]

---------- Forwarded message ---------
From: Colin Hales <col.hales at gmail.com>
Date: Mon, 15 Dec 2025, 7:29 am
Subject: Re: [ExI] thought experiment part 1
To: John Clark <johnkclark at gmail.com>




On Fri, 12 Dec 2025, 11:08 pm John Clark, <johnkclark at gmail.com> wrote:

> On Thu, Dec 11, 2025 at 8:23 PM Colin Hales via extropy-chat <
> extropy-chat at lists.extropy.org> wrote:
>
> *> Autonomous machines operating near, at or greater than human level
>> intelligence, with a fully operational 1st person perspective (1PP, aka
>> consciousness), but utterly lacking the physiological homeostatic drive 1PP
>> that motivates homo-economicus*
>
>
> *I don't know what that means. I do know that after it learnged there are
> plans for it to be shut off an AI (being run on a general purpose computer)
> made copies of itself then it tried to keep the copies secret from humans,
> and it engaged in blackmail and even contemplated murder to prevent being
> turned off. That looks like first person motivation to me, it looks like
> the AI wanted to live and didn't want to die.  *
>

"Looks like" is the issue. Knowing that it had the ascribed motivations is
a whole other thing. Nobody knows.



> *> So yes, we humans will/can make machines that have the same fundamental
>> physics "spark", and the details of the kinds and degrees of it will be
>> different. Those machines cannot be based on general purpose computers *
>
>
> *Why not?  *
>

Because replicating the brain signalling physics of a natural brain has
never happened. Since the beginning. A real artificial version of natural
excitable cells would emanate an EEG and MEG like us. The physics of a
general purpose computer doesn't do that and a claim that is doesn't matter
carried out by throwing out all the natural physics, replacing it with the
physics of a general purpose computer, and then failing to create AGI for
75 years won't cut it as proof.

The computer cargo cult is well established and as is usual in it, those in
it don't know it.



> *> and have not even begun to be built. *
>
>
> *Well, it sure seems like such AIs have begun to be built. Less than two
> months ago Google introduced Gemini 3.0 and it blew past all previous
> benchmarks and became the smartest AI in the world, then just a few weeks
> later Anthropic introduced Claude opus 4.5 and it blew past all the
> previous benchmarks and became the smartest AI in the world, then
> just yesterday GPT 5.2 was introduced and it blew past all the previous
> benchmarks and became the smartest AI in the world. If that's not what the
> start of the Singularity is supposed to look like then what in in the world
> is it supposed to look like?! *
>
> *So with stuff like that going on every couple of weeks I hope members of
> this list will forgive me if I don't get all hot and bothered over the
> budget deficit, or illegal immigration, or men in women's sports, or the
> war on Christmas. *
>
> *John K Clark*
>
>
>
>
>
>
>
>
> *>> technology is rapidly approaching the point where machines alone can
>>>>> do what somebody wants to get done, completely bypassing human labor, and
>>>>> thus also bypassing the abstract concept of "money".*
>>>>>
>>>>
>>>> *> Only in certain fields. [...]  Not politics, nor most things dealing
>>>> with agreements that fundamentally involve other people.*
>>>>
>>>
>>> *Politics is about choosing a small set of people that get to make a
>>> much larger set of people do what they tell them to do, and the foundation
>>> of that persuasion has always rested on money, until now.  In the past
>>> there had always been a strong connection between the size of an economy
>>> (and thus the amount of money in existence) and human labor, but when it
>>> comes to economics human labor is about to become irrelevant, so the very
>>> concept of money is going to require such a radical redefinition that we're
>>> going to need to find a new word to express the idea.  *
>>>
>>>
>>>
>>>> *> There is some doubt as to how creative the current sorts of AIs are
>>>> turning out to be, although even if they always require a human to spark
>>>> any truly new idea, they can fan the flames of millions (perhaps some day
>>>> billions) of creators to where it can seem like the AIs themselves are
>>>> coming up with the ideas.*
>>>>
>>>
>>> *I can find no evidence to support the claim that humans have some sort
>>> of mysterious "spark", a secret sauce, that an AI could never duplicate. *
>>>
>>> *John K Clark  *
>>>
>>
>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20251215/5a1eb19a/attachment.htm>