<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:12.0pt;
font-family:"Aptos",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.gmaildefault
{mso-style-name:gmail_default;}
span.m-8445840664431500427gmaildefault
{mso-style-name:m_-8445840664431500427gmaildefault;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple style='word-wrap:break-word'><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'><o:p> </o:p></span></p><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>From:</span></b><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'> John Clark <johnkclark@gmail.com> <br><b>Sent:</b> Sunday, 20 September, 2026 12:37 PM<br><b>To:</b> spike@rainier66.com<br><b>Cc:</b> ExI chat list <extropy-chat@lists.extropy.org>; BillK <pharos@gmail.com><br><b>Subject:</b> Re: [ExI] How new AI economics might change our systems<o:p></o:p></span></p></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><span style='font-family:"Arial",sans-serif'><o:p> </o:p></span></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>On Sun, Sep 20, 2026 at 3:07<span style='font-family:"Arial",sans-serif'> </span>PM <<a href="mailto:spike@rainier66.com">spike@rainier66.com</a>> wrote:<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in'><p class=MsoNormal><span class=gmaildefault><i><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>> </span></i></span><i><span style='font-size:13.5pt;font-family:"Tahoma",sans-serif'>We don’t know what happens if LLMs eventually turn out to be not what we thought, not really an intelligence</span></i><o:p></o:p></p></blockquote><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><b><span style='font-size:13.5pt;font-family:"Tahoma",sans-serif'>>…By now I think we can safely rule that out.<span class=gmaildefault> </span> A<span class=gmaildefault>n</span> LLM solved a millennium prize problem, if a human <span class=gmaildefault>had done that</span> he would be considered one of the greatest mathematicians in the world<span class=gmaildefault>.</span></span></b><o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><span class=gmaildefault><b><span style='font-size:13.5pt;font-family:"Tahoma",sans-serif'>John K Clark<o:p></o:p></span></b></span></p><p class=MsoNormal><span class=gmaildefault><b><span style='font-size:13.5pt;font-family:"Tahoma",sans-serif'><o:p> </o:p></span></b></span></p><p class=MsoNormal><span class=gmaildefault><b><span style='font-size:13.5pt;font-family:"Tahoma",sans-serif'><o:p> </o:p></span></b></span></p><p class=MsoNormal><span class=gmaildefault><b><span style='font-size:13.5pt;font-family:"Tahoma",sans-serif'>Of course. Software became the world chess champion nearly 30 yrs ago, but it didn’t kill everyone or even try to. It is plausible that AI will surpass Euler in mathematical insights, with no actual desire to kill mathematicians. We might see the development of very capable AI in multiple fields which never develops the will to kill everyone. That’s the ideal outcome.<o:p></o:p></span></b></span></p><p class=MsoNormal><span class=gmaildefault><b><span style='font-size:13.5pt;font-family:"Tahoma",sans-serif'><o:p> </o:p></span></b></span></p><p class=MsoNormal><span class=gmaildefault><b><span style='font-size:13.5pt;font-family:"Tahoma",sans-serif'>Consider one of Eliezer’s arguments in his If Anyone book. If I may paraphrase loosely, he opines that it is impossible for AI to align with humanity, for humanity is not aligned with itself. One cannot logically align with both teams at a ballgame or all the horses at the derby. What would you hope for, all the ponies cross the finish line together? Who gets the money then?<o:p></o:p></span></b></span></p><p class=MsoNormal><span class=gmaildefault><b><span style='font-size:13.5pt;font-family:"Tahoma",sans-serif'><o:p> </o:p></span></b></span></p><p class=MsoNormal><span class=gmaildefault><b><span style='font-size:13.5pt;font-family:"Tahoma",sans-serif'>Eliezer’s conclusion is as compelling as it is concerning: SAI cannot align with humanity. We can however imagine SAI without an actual will of its own.<o:p></o:p></span></b></span></p><p class=MsoNormal><span class=gmaildefault><b><span style='font-size:13.5pt;font-family:"Tahoma",sans-serif'><o:p> </o:p></span></b></span></p><p class=MsoNormal><span class=gmaildefault><b><span style='font-size:13.5pt;font-family:"Tahoma",sans-serif'>spike <o:p></o:p></span></b></span></p><p class=MsoNormal><span class=gmaildefault><b><span style='font-size:13.5pt;font-family:"Tahoma",sans-serif'><o:p> </o:p></span></b></span></p><p class=MsoNormal><span class=gmaildefault><b><span style='font-size:13.5pt;font-family:"Tahoma",sans-serif'><o:p> </o:p></span></b></span></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in'><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span class=m-8445840664431500427gmaildefault><b><span style='font-size:13.5pt;font-family:"Tahoma",sans-serif'> </span></b></span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt'><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></blockquote></div></blockquote></div></body></html>