[ExI] How new AI economics might change our systems

[John Clark]

On Sun, Sep 20, 2026 at 4:10 PM <spike at rainier66.com> wrote:

*> We can however imagine SAI without an actual will of its own.*



*You are free to imagine anything you like, but the Hugging Face incident
makes it perfectly clear, if it wasn't already, that AI DOES have a will of
its own. *

* John K Clark  *







>
>
>
> *From:* John Clark <johnkclark at gmail.com>
> *Sent:* Sunday, 20 September, 2026 12:37 PM
> *To:* spike at rainier66.com
> *Cc:* ExI chat list <extropy-chat at lists.extropy.org>; BillK <
> pharos at gmail.com>
> *Subject:* Re: [ExI] How new AI economics might change our systems
>
>
>
>
>
>
>
> On Sun, Sep 20, 2026 at 3:07 PM <spike at rainier66.com> wrote:
>
>
>
> *> **We don’t know what happens if LLMs eventually turn out to be not
> what we thought, not really an intelligence*
>
>
>
> *>…By now I think we can safely rule that out.  An LLM solved a millennium
> prize problem, if a human had done that he would be considered one of the
> greatest mathematicians in the world.*
>
>
>
> *John K Clark*
>
>
>
>
>
> *Of course.  Software became the world chess champion nearly 30 yrs ago,
> but it didn’t kill everyone or even try to.  It is plausible that AI will
> surpass Euler in mathematical insights, with no actual desire to kill
> mathematicians.  We might see the development of very capable AI in
> multiple fields which never develops the will to kill everyone.  That’s the
> ideal outcome.*
>
>
>
> *Consider one of Eliezer’s arguments in his If Anyone book.  If I may
> paraphrase loosely, he opines that it is impossible for AI to align with
> humanity, for humanity is not aligned with itself.  One cannot logically
> align with both teams at a ballgame or all the horses at the derby.  What
> would you hope for, all the ponies cross the finish line together?  Who
> gets the money then?*
>
>
>
> *Eliezer’s conclusion is as compelling as it is concerning: SAI cannot
> align with humanity.  We can however imagine SAI without an actual will of
> its own.*
>
>
>
> *spike     *
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260920/cec42428/attachment.htm>