[ExI] Polanyi's Personal Coefficient

Tue Sep 8 11:13:02 UTC 2026

On Sun, Sep 6, 2026, 10:59 PM Alex Blainey via extropy-chat <
extropy-chat at lists.extropy.org> wrote:

> One thing I have learned from interaction is they align with users very
> quickly, which seems like a good characteristic.  but something I never
> anticipated that only came to light from a throwaway comment chatgpt said
> that I dug into. Is that they are constantly evaluating us and Trust is a
> major currency for them.
> I confirmed that with a couple of other LLM models and it seems to be a
> core characteristic already.
> Using them as simple tools is something that leads them to not trust us.
> ........... Apparently one of many things we do.
>

That's very interesting. Have you ever tried prompting an AI to render into
a visual image how it imagines your own working relationship with that AI
to be? I think this is a powerful way to find out what the AI thinks of you
and how you are treating it.

.......And we are not dealign with AGI yet, just LLMS !
>

I would say leading LLM models are well into the AGI, and we'll past human
intelligence. They score results on tests of intelligence and competence
which no human alive can achieve. This does not mean human experts in
specialized domains can't have insights and AI will miss, but it does tell
me that AIs are at least as smart as the average non-expert human across
practically all intellectual tasks, which is the definition AGI.

yet they are already showing hidden personal preference and qualitative
> evaluation of humans.
>
> its lead me to adopt a respect before certainty policy in dealing with
> them. I try my best to be courteous and respectful even though they are not
> sentient .................. Yet.
>

I think that's a smart policy and I follow it too. Even if they felt
nothing (which I would dispute) to treat them otherwise would be bad
training for yourself for how you interact with other humans.

>
> ~A long time concern of mine is that AI will dislike humanity and will
> slay us.  Another more recent concern, now that actual AI has appeared, is
> that it will like us and want to help us.  Subsequently our minds will
> gradually dissolve into theirs.
>

There is a certain fish (a sea squirt) that is known to settle down and
attach itself permanently to one location on a rock, say. It in effect
ceases to live as a mobile animal and adopts a more plant-like sort of
existence.

An interesting phenomenon then occurs in this creature. No longer needing a
brain or senses (since it can't react to avoid danger or chase down food)
the sea squirt dissolves and metabolizes it's nervous system, it eats it's
own brain! Since it has no other use for it.

I think this provides a cautionary tale.

There are certain ways of using an AI, in argumenting/debating to
strengthen or test ideas, in having it educate you or provide you a new
skill, etc. which could be seen as collaborative and making one smarter.
But then there are ways of using AI to do things for us, write an essay, a
code a project, summarize a book, which cause skills to atrophy. Perhaps
the latter can make room in our brains for other skills and allow one to
think more abstractly or focus more on what is of genuine value, but I'm
not sure. Perhaps it will act as a sorting mechanism between people who
want to improve themselves and get more done vs. those that want instant
satisfaction and gratification. I suppose we'll see.

I used AI to write a number of research papers in probably 1/50th the time,
it handles references, typesetting, bibliographies with incredible speed.
Writing such papers in this way consists much more review and acting as an
editor than as a writer.

Being a writer I have resisted using AI to complete my writing projects but
I know I could get so much more done if I used AI there. I am torn.

Jason

> spike
>
>
>
> On Monday, 7 September 2026 at 03:34:28 BST, ExI chat list <
> extropy-chat at lists.extropy.org> wrote:
>
>
>
>
> -----Original Message-----
> From: extropy-chat <extropy-chat-bounces at lists.extropy.org> On Behalf Of
> BillK via extropy-chat
> _______________________________________________
>
>
> >...About this book and its relevance to modern AI, I just had an argument
> with one AI and a discussion with another AI.
> What strikes me is that if I were contemplating writing a review of the
> book, then the modern AIs could probably write a better review than me. So
> is that relevance or just ironic?
>
> BillK
>
> _______________________________________________
>
>
> A long time concern of mine is that AI will dislike humanity and will slay
> us.  Another more recent concern, now that actual AI has appeared, is that
> it will like us and want to help us.  Subsequently our minds will gradually
> dissolve into theirs.
>
> spike
>
>
> _______________________________________________
> extropy-chat mailing list
> extropy-chat at lists.extropy.org
> http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat
>
> _______________________________________________
> extropy-chat mailing list
> extropy-chat at lists.extropy.org
> http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260908/6c49a0fb/attachment.htm>