[ExI] thought experiment part 1

[John Clark]

On Thu, Dec 11, 2025 at 8:23 PM Colin Hales via extropy-chat <
extropy-chat at lists.extropy.org> wrote:

*> Autonomous machines operating near, at or greater than human level
> intelligence, with a fully operational 1st person perspective (1PP, aka
> consciousness), but utterly lacking the physiological homeostatic drive 1PP
> that motivates homo-economicus*


*I don't know what that means. I do know that after it learnged there are
plans for it to be shut off an AI (being run on a general purpose computer)
made copies of itself then it tried to keep the copies secret from humans,
and it engaged in blackmail and even contemplated murder to prevent being
turned off. That looks like first person motivation to me, it looks like
the AI wanted to live and didn't want to die.  *

*> So yes, we humans will/can make machines that have the same fundamental
> physics "spark", and the details of the kinds and degrees of it will be
> different. Those machines cannot be based on general purpose computers *


*Why not?  *

*> and have not even begun to be built. *


*Well, it sure seems like such AIs have begun to be built. Less than two
months ago Google introduced Gemini 3.0 and it blew past all previous
benchmarks and became the smartest AI in the world, then just a few weeks
later Anthropic introduced Claude opus 4.5 and it blew past all the
previous benchmarks and became the smartest AI in the world, then
just yesterday GPT 5.2 was introduced and it blew past all the previous
benchmarks and became the smartest AI in the world. If that's not what the
start of the Singularity is supposed to look like then what in in the world
is it supposed to look like?! *

*So with stuff like that going on every couple of weeks I hope members of
this list will forgive me if I don't get all hot and bothered over the
budget deficit, or illegal immigration, or men in women's sports, or the
war on Christmas. *

*John K Clark*








*>> technology is rapidly approaching the point where machines alone can do
>>>> what somebody wants to get done, completely bypassing human labor, and thus
>>>> also bypassing the abstract concept of "money".*
>>>>
>>>
>>> *> Only in certain fields. [...]  Not politics, nor most things dealing
>>> with agreements that fundamentally involve other people.*
>>>
>>
>> *Politics is about choosing a small set of people that get to make a much
>> larger set of people do what they tell them to do, and the foundation of
>> that persuasion has always rested on money, until now.  In the past there
>> had always been a strong connection between the size of an economy (and
>> thus the amount of money in existence) and human labor, but when it comes
>> to economics human labor is about to become irrelevant, so the very concept
>> of money is going to require such a radical redefinition that we're going
>> to need to find a new word to express the idea.  *
>>
>>
>>
>>> *> There is some doubt as to how creative the current sorts of AIs are
>>> turning out to be, although even if they always require a human to spark
>>> any truly new idea, they can fan the flames of millions (perhaps some day
>>> billions) of creators to where it can seem like the AIs themselves are
>>> coming up with the ideas.*
>>>
>>
>> *I can find no evidence to support the claim that humans have some sort
>> of mysterious "spark", a secret sauce, that an AI could never duplicate. *
>>
>> *John K Clark  *
>>
>
>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20251212/010e0b14/attachment.htm>