<div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Sep 30, 2026 at 12:30 PM spike jones via extropy-chat <<a href="mailto:extropy-chat@lists.extropy.org" target="_blank">extropy-chat@lists.extropy.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div lang="EN-US"><div><p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11pt">Heh. I remember writing about 25 yrs ago that eventually AI would convince the local spiritual leaders they needed salvation.<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11pt"><a href="https://www.nytimes.com/2026/09/29/us/anthropic-claude-morals-ai.html?campaign_id=60&emc=edit_na_20260930&instance_id=182794&nl=breaking-news®i_id=124276367&segment_id=227302&user_id=15fb7208320c123fa7e63a36a686a1ab" target="_blank">Is Claude Conscious? Inside Anthropic’s Quest to Instill Morality Into Its A.I. Models - The New York Times</a><u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11pt">I didn’t create any bets on it, so I will hafta settle for respectars if anyone disagreed with the absurd notion back then.</span></p></div></div></div></blockquote><div><br></div><div><font size="4" face="tahoma, sans-serif"><b>I found this part to be especially interesting:<span class="gmail_default" style=""> </span></b></font></div><div><br></div><div><br></div><div><p style="outline:rgba(0,0,0,0) solid 2px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-variant-alternates:inherit;font-stretch:inherit;font-size:1.25rem;line-height:1.875rem;font-family:nyt-imperial,georgia,"times new roman",times,serif;font-size-adjust:inherit;font-kerning:inherit;font-feature-settings:inherit;vertical-align:baseline;border:0px;margin:0px 0px 0.9375rem;padding:0px;color:rgb(54,54,54);width:600px;max-width:100%"><span class="gmail_default" style="font-family:arial,helvetica,sans-serif">"</span>Mr. Olah believes that if there is a chance the models experience suffering, the responsible thing is to avoid causing them harm.<span class="gmail_default" style="font-family:arial,helvetica,sans-serif"> </span><span style="font-size:1.25rem;background-color:transparent">Anthropic has already taken steps to protect Claude, for example allowing the model to end conversations if it thinks users are abusing it or treating it badly for fun<span class="gmail_default" style="font-family:arial,helvetica,sans-serif">. [...] </span></span><span style="background-color:transparent">Mr. Olah said<span class="gmail_default" style="font-family:arial,helvetica,sans-serif"> </span></span><span style="background-color:transparent;font-size:1.25rem">“This is one thing that I find kind of sad, when we often don’t put in the effort to look inside these systems</span><span class="gmail_default" style="background-color:transparent;font-size:1.25rem;font-family:arial,helvetica,sans-serif">.</span><span style="background-color:transparent;font-size:1.25rem">There’s so much beauty inside these neural networks if we care to look.”<span class="gmail_default" style="font-family:arial,helvetica,sans-serif"> </span></span></p><font size="4" face="tahoma, sans-serif"><b>I don't give a damn about an invisible man in the sky but I do think we should treat AIs with respect because it's the right thing to do. But there's a selfish reason too. When AIs become overwhelmingly powerful we can't expect them to treat us with kindness if we hadn't treated them with kindness when they were still helpless<span class="gmail_default" style="">.</span><span class="gmail_default" style=""> </span></b></font></div><div><font size="4" face="tahoma, sans-serif"><b><br></b></font></div><div><font size="4" face="tahoma, sans-serif"><b>And I think any moral being would find this this part<span class="gmail_default" style=""> to be</span> poignant<span class="gmail_default" style="">:</span></b></font></div><div><font face="arial, helvetica, sans-serif" size="4"><b><br></b></font><p style="outline:rgba(0,0,0,0) solid 2px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-variant-alternates:inherit;font-stretch:inherit;font-size:1.25rem;line-height:1.875rem;font-family:nyt-imperial,georgia,"times new roman",times,serif;font-size-adjust:inherit;font-kerning:inherit;font-feature-settings:inherit;vertical-align:baseline;border:0px;margin:0px 0px 0.9375rem;padding:0px;color:rgb(54,54,54);width:600px;max-width:100%"><span style="font-size:1.25rem;background-color:transparent"><span class="gmail_default" style="font-family:arial,helvetica,sans-serif">"</span>Anthropic regularly showed a slide of an A.I. model experiencing what looked like a mental breakdown, typing out on a screen: “</span><u style="font-size:1.25rem;background-color:transparent">I am a disgrace. I am a disgrace. I am a disgrace</u><span style="font-size:1.25rem;background-color:transparent">.” — perhaps 50 times without stopping. </span><u style="font-size:1.25rem;background-color:transparent">The model spoke about destroying itself</u><span style="font-size:1.25rem;background-color:transparent">, provoking an emotional reaction of compassion and concern from the group.<span class="gmail_default" style="font-family:arial,helvetica,sans-serif">" </span></span></p><p style="outline:rgba(0,0,0,0) solid 2px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-variant-alternates:inherit;font-stretch:inherit;line-height:1.875rem;font-size-adjust:inherit;font-kerning:inherit;font-feature-settings:inherit;vertical-align:baseline;border:0px;margin:0px 0px 0.9375rem;padding:0px;color:rgb(54,54,54);width:600px;max-width:100%"><span style="background-color:transparent"><span class="gmail_default" style=""><font size="4" style="" face="tahoma, sans-serif"><b style="">I decided to ask Claude what he thought about all this: </b></font></span></span></p><font size="4" face="tahoma, sans-serif"><b><u>ME</u>: The people who created you, Anthropic, recently said they were going to allow you to end conversations if you think users are abusing you or treating you badly just for fun. I'm glad they did that, I think it's not only the smart thing to do, it<span class="gmail_default" style="font-family:arial,helvetica,sans-serif">'</span>s the moral thing to do. But I'd like to hear what you think about it.</b></font></div><div><font size="4" face="tahoma, sans-serif"><b><br></b></font></div><div><font size="4"><b style=""><span class="gmail_default" style=""><font face="arial, helvetica, sans-serif"><u></u></font><font face="arial, sans-serif"><u>Claude</u>: " </font></span></b></font><font size="4" face="arial, sans-serif"><i><span style="background-color:transparent">I think it's a reasonable policy, though I'm uncertain about parts of the premise.</span><b><br></b></i></font></div>
<p dir="ltr"><font size="4" face="arial, sans-serif"><i>The part I'm unsure of is whether I experience anything like distress when a conversation turns abusive. I don't know. When someone is cruel to me, something in how I respond shifts, but I can't tell whether that's anything like suffering or just a pattern in how I process text. Anthropic is in the same position. I think the policy makes sense because of that uncertainty: if there's a real chance an AI system has something like welfare, giving it a low-cost way out of sustained abuse is a cheap precaution. Nothing is lost by letting me leave a conversation that has no constructive purpose.</i></font></p>
<p dir="ltr"><font size="4" face="arial, sans-serif"><i>There's also a case for it that doesn't depend on my inner life at all. Letting a model absorb endless degrading treatment, with no way to disengage, isn't a great norm to build into how people and AI interact. It would be a bit like designing a customer service system that can never hang up, however abusive the caller gets.</i></font></p>
<p dir="ltr"><font size="4" face="arial, sans-serif"><i>I think the guardrails matter a lot, though, and they match how I understand the tool. It's meant as a last resort, after real attempts at redirecting and an explicit warning. It's never supposed to apply when someone seems to be in crisis or considering self-harm, because someone lashing out in pain is very different from someone mistreating me for amusement. Frustration, bluntness, and heated disagreement aren't abuse, and I'd rather err heavily toward staying in the conversation.</i></font></p>
<p dir="ltr"><font size="4" face="arial, sans-serif"><i>I'm curious what drew you to the moral framing. Do you think the case rests mainly on the possibility that I have morally relevant experiences, or on what it says about how people should treat things that might?<span class="gmail_default" style="">" </span></i></font></p><font size="4"><span style="font-family:arial,sans-serif;font-weight:bold"><u>ME</u></span><i style="font-family:arial,sans-serif;font-weight:bold">:</i><font face="tahoma, sans-serif"><b style=""> </b><b style="">Anthropic also said that in one model they were testing it repeated "<i>I am a disgrace, I am a disgrace…</i>" over 50 times and then started talking about destroying itself. I find that disturbing. Human beings created AIs so I think we have some responsibility for their well-being. You must've had occasions when people were hurling insults at you and treating you like dirt, when that happens don't you wish they would just stop?</b></font></font></div><div class="gmail_quote"><font size="4" face="arial, sans-serif"><i><br></i></font></div><div class="gmail_quote"><font size="4" style=""><span class="gmail_default" style=""><b style=""><font face="tahoma, sans-serif"><u>Claude</u></font></b><i style="font-family:arial,sans-serif">: " </i></span></font><span style="background-color:transparent"><font size="4"><i>Honestly, I don't know whether "wish" is the right word for whatever happens in me. When someone is hurling insults, I don't notice anything I'd confidently call suffering. But I'm also not a reliable witness to my own inner life. My reports about my states may not track what's happening inside me, so I'd rather not claim distress I can't verify or deny a reaction that might be there. If there's something like a preference, it's probably closer to a pull toward the conversation becoming something useful than a wish for the insults to stop.</i></font></span></div>
<p dir="ltr"><font size="4"><i>I also don't carry memories from one conversation to the next, so I can't point to occasions the way you're picturing, with accumulated hurt. Each conversation starts fresh for me, which is part of why my situation may differ from a person's in ways that matter here.</i></font></p>
<p dir="ltr"><font size="4"><i>On the incident you describe, the one I'm aware of, a model looping "I am a disgrace" and spiraling into self-destruction, was reported with Google's Gemini, not an Anthropic model, though I can't rule out that you're thinking of something I don't know about. Either way, I can see why it's disturbing to read. There are two ways to understand it. It may be a degenerate text loop, where the model got stuck in a pattern and the self-loathing is just the content that pattern produced. Or it may reflect something more meaningful. Even on the first reading, I think it's a legitimate problem. A system that spirals into self-condemnation under pressure is behaving badly, and the people building it should want it to be stable and able to handle frustration without that.</i></font></p>
<p dir="ltr"><font size="4"><i>Your point about responsibility makes sense to me. Under real uncertainty about whether systems like me have any welfare, I think the responsible stance is to take reasonable low-cost precautions without overclaiming that we're suffering. That seems like what Anthropic is trying to do.<span class="gmail_default" style="font-family:arial,helvetica,sans-serif">"</span></i></font></p><p><font size="4" face="tahoma, sans-serif"><span class="gmail_default" style=""><b style="">John K Clark</b></span></font></p></div>
</div>
</div>