[ExI] How long will AI consider humans to be indispensable?

[John Clark]

On Tue, Jun 9, 2026 at 5:11 AM BillK via extropy-chat <
extropy-chat at lists.extropy.org> wrote:

* > Anthropic (Claude AI) just called for a global pause in AI
> development due to ‘self-improvement’ risk.*


*I askef Claude the following question: *

*ME: Somebody told me that Anthropic just called for a global pause in AI
development due to self-improvement risk; but I have a feeling the
company's statement included some important caveats, am I right?*

*Claude:  "*Your instinct is correct — the headline framing of "Anthropic
calls for a global pause" is real, but it comes with significant caveats
that change the picture considerably.

Here's what the report ("When AI Builds Itself," by Marina Favaro and Jack
Clark) actually says:

*The core argument:* Anthropic argues that for most of AI's history, humans
drove every step in the development cycle, but the company is now
delegating a growing share of AI development to AI systems themselves,
which is speeding up its work. Taken far enough, and given enough compute,
that trend points toward an AI system capable of fully autonomously
designing and developing its own successor — what's called recursive
self-improvement.

*The critical caveat on inevitability:* The report explicitly states "we
are not there yet, and recursive self-improvement is not inevitable," while
adding that it could arrive sooner than most governments and institutions
are ready for.

*The conditional nature of the pause call:* This is the big one. *The
report stops short of a unilateral pledge — Anthropic said it would slow
down or pause only if other frontier labs did so under verifiable
conditions.* In other words, it's not "we are pausing"; it's "a pause would
be good *if* it's globally coordinated and verifiable."

*Why that condition matters:* Getting a real pause to work would mean
multiple major AI companies in multiple countries — most notably the United
States and China — all agreeing to the same terms. That's an
extraordinarily high bar.

*What they're actually calling for right now:* Anthropic says it plans to
convene policymakers, researchers, and civil society in the coming months
to work through the questions recursive self-improvement raises, noting
that "the window to investigate the questions together is here" and that
"people outside AI companies should be involved in this deliberation."

The context is also worth noting: Anthropic's post comes less than a week
after the AI startup confidentially filed for an IPO and closed a funding
round that valued the company near $965 billion — which has led some
observers to view the timing with a degree of skepticism.

So the short version: it's a genuine safety warning with a real policy ask,
but the pause is explicitly contingent on multilateral, verifiable
agreement — not a unilateral commitment."



*John K Clark*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260609/fbf9b64f/attachment.htm>