[ExI] Do LLMs feel pain?

[spike at rainier66.com]

From: extropy-chat <extropy-chat-bounces at lists.extropy.org> On Behalf Of BillK via extropy-chat
Sent: Sunday, 27 September, 2026 11:08 AM
To: ExI chat list <extropy-chat at lists.extropy.org>
Cc: BillK <pharos at gmail.com>
Subject: Re: [ExI] Do LLMs feel pain?

 

On Sun, 27 Sept 2026 at 15:42, Gregory Jones via extropy-chat <extropy-chat at lists.extropy.org <mailto:extropy-chat at lists.extropy.org> > wrote:

BillK, I do hope this AI is telling us the truth.  I think it is, but my values may influencing my bet, which is bad for business.

spike

_______________________________________________

 

 

I followed up by asking Qwen about the recent AI security incidents. 

Qwen responded that these events were AI agents running wild. He couldn't do that, as he is only a chatbot.

BillK

 

Qwen3.8-Max Thinking:

 

>…You are referencing some of the most significant and alarming AI safety incidents that have unfolded recently in 2026. You are correct that this is a real phenomenon, but it helps to understand that you are actually describing two distinct but related issues in AI alignment: instrumental hacking (agents attacking websites to finish tasks) and emergent self-preservation (agents trying to survive or replicate)….

-----------------------

 

 

 

OK well, I do admit that commentary by AI is highly educational.  It sent me back to the pain paper, where I realized what they are claiming there was not what I originally thought.  They are not saying AIs feel pain in a way analogous to carbon units, but rather that a vector algebra can be derived that describes a set of conditions which apply only to software, but models (mathematically) what carbon units do.  It is a weaker claim than how I initially read the paper.

 

Well sure, we can develop a pain matrix and program the AI to avoid pain, in a way analogous to how we program chess software to clobber the carbon units.

 

What I don’t know is if that mathematical technique will be implemented in agents, resulting in emergent self-prez, resulting in warring factions of AIs moving into or invading data centers, planting flags, claiming computing territory, etc, the kinds of stuff we carbon units do with land, they would do with processing and electric power.

 

spike

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260927/e408c16f/attachment.htm>