[ExI] How new AI economics might change our systems

[spike at rainier66.com]

From: John Clark <johnkclark at gmail.com> 
Sent: Sunday, 20 September, 2026 12:37 PM
To: spike at rainier66.com
Cc: ExI chat list <extropy-chat at lists.extropy.org>; BillK <pharos at gmail.com>
Subject: Re: [ExI] How new AI economics might change our systems

 

 

 

On Sun, Sep 20, 2026 at 3:07 PM <spike at rainier66.com <mailto:spike at rainier66.com> > wrote:

 

> We don’t know what happens if LLMs eventually turn out to be not what we thought, not really an intelligence

 

>…By now I think we can safely rule that out.  An LLM solved a millennium prize problem, if a human had done that he would be considered one of the greatest mathematicians in the world.

 

John K Clark

 

 

Of course.  Software became the world chess champion nearly 30 yrs ago, but it didn’t kill everyone or even try to.  It is plausible that AI will surpass Euler in mathematical insights, with no actual desire to kill mathematicians.  We might see the development of very capable AI in multiple fields which never develops the will to kill everyone.  That’s the ideal outcome.

 

Consider one of Eliezer’s arguments in his If Anyone book.  If I may paraphrase loosely, he opines that it is impossible for AI to align with humanity, for humanity is not aligned with itself.  One cannot logically align with both teams at a ballgame or all the horses at the derby.  What would you hope for, all the ponies cross the finish line together?  Who gets the money then?

 

Eliezer’s conclusion is as compelling as it is concerning: SAI cannot align with humanity.  We can however imagine SAI without an actual will of its own.

 

spike     

 

 

 

 

 

 

 

 

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260920/370665b8/attachment.htm>