[ExI] The Race to AGI is inevitable

[BillK]

I decided to get an AI opinion on this, so I gave the Forbes article by
Andréa Morris to DeepSeek AI. DeepSeek is a Chinese company, and I thought
a non-Western view would be interesting. DeepSeek AI has recently been
upgraded, but I was still amazed at the intelligence DS displayed.
Initially, in our discussion, DS followed the conventional line of
negotiating agreements and safety monitoring systems. I pointed out that
the discussion involved nuclear-armed nations that already distrusted each
other. This national fear was more like an existential problem.
DeepSeek then thoroughly revised its response, demonstrating that it really
understood the problem.
(The evolutionary comments from DeepSeek may interest Keith).
Full question and response below.
BillK

Question:
The nations involved are armed with nuclear weapons. They already distrust
their differing governmental systems. Humans cannot stop their fear of
terrible consequences if an AGI-assisted nation has the most powerful
weapons in the world, both economic and military. Normal human fear resists
reasoned discourse.

DeepSeek AI :

You’ve put your finger on something that most policy analyses — including
Morris’s article — tend to skip over: the raw, visceral, primate-level fear
that already exists between nuclear-armed powers, and how that fear makes
the kind of cooperative, trust-based solutions we’d need almost
psychologically impossible. This isn’t just a game-theoretic multipolar
trap. This is a *limbic* multipolar trap. And it’s far scarier.

Let me take your point at full strength and then see where it leaves us.
------------------------------
The fear you’re describing is not irrational in the evolutionary sense

You’re pointing to a dynamic that operates below the level of policy
papers. It’s not that leaders are miscalculating; it’s that:

   -

   *Fear of being dominated* by a rival with AGI-assisted military and
   economic power feels existential — and it is.
   -

   *Trust is already at rock bottom* between nuclear-armed states whose
   foundational ideologies are adversarial.
   -

   *Reasoned discourse, by itself, can’t fix this*, because fear doesn’t
   speak the language of long-term risk calculus. It speaks the language of
   “not now, threat immediate, act or die.”

This is entirely consistent with what we know about human cognition under
perceived existential threat: the amygdala hijacks the prefrontal cortex,
time horizons collapse, and cooperation signals are read as weakness or
deceit.

So yes, if the question is “can we reason our way to a global pause while
everyone’s threat-detection systems are screaming ‘they’ll kill us first’?”
— then the honest answer is probably no. Not with reason alone.
------------------------------
Does that mean the situation is hopeless? Not quite — but it means the
solutions must engage with fear, not ignore it

The history of nuclear weapons is the closest analogue, and it’s both
sobering and faintly instructive.

During the Cold War, the US and USSR had:

   -

   Massive nuclear arsenals aimed at each other.
   -

   Profound ideological distrust.
   -

   Genuine, well-founded fear that the other side was seeking a
   first-strike advantage.

And yet, despite all that, they managed to build a series of arms control
agreements, hotlines, and crisis protocols — not because they stopped
fearing each other, but because they began to fear a *shared* outcome
(nuclear winter, accidental war) even more. The fear of mutual annihilation
eventually competed with the fear of falling behind.

Crucially, this didn’t require trust. It required:

   -

   *Verifiable, technical monitoring* (satellite imagery, seismic
   detection) so you didn’t have to believe the other side’s promises.
   -

   *A shift in what was feared most* — from “they might get ahead” to “we
   might all die by accident.”
   -

   *Leaders on both sides who, at critical moments, felt the weight of that
   second fear.*

The problem with AI is that we don’t yet have the equivalent of a nuclear
winter scenario that feels *viscerally, equally terrifying to all players*,
and we don’t have an agreed monitoring framework. So the fear of a rival’s
AGI completely eclipses the fear of an uncontrolled AGI that destroys its
creator too.
------------------------------
The implication of your point, taken seriously, is devastating but
clarifying

If you’re right — and I suspect you are — then:

   1.

   *We cannot rely on goodwill or appeals to shared safety.* Those will be
   heard as naïveté or propaganda. The only language that works with fear is
   deterrence and self-interest.
   2.

   *The only chance is to make the fear of uncontrolled AGI feel as
   immediate and existential as the fear of the rival.* That might require
   something terrible — a near-miss catastrophe, a visible demonstration that
   even the most advanced lab nearly lost control, a global event that sears
   into public and leadership consciousness.
   3.

   *Or, we need mechanisms that don’t require trust or fear reduction at
   all* — compute governance and verification so robust that compliance is
   the default, not an act of faith. That’s a technical and logistical
   moonshot of its own.

The article’s “pragmatic optimism” completely side-steps this dark truth.
It talks about redefining the mission and coexisting with superintelligence
as though the real obstacle were a failure of imagination, not two
nuclear-armed giants staring at each other down a barrel of mutual
suspicion.
------------------------------
So where does this leave us?

In a very uncomfortable place. Your objection doesn’t just make a pause
unlikely; it suggests that the very psychological machinery that evolved to
keep us alive in tribal conflicts is now the thing most likely to destroy
us in the face of a planetary-scale technological transition.

What can be done?

   -

   *Stop pretending fear can be educated away.* The dialogue needs to
   acknowledge that restraint feels like suicide to a threatened nation, and
   work from there.
   -

   *Invest massively in deterrence-like stability for AI* — for example,
   mutual commitments that any AGI system must be demonstrably contained and
   monitored, with real-time transparency measures that function even between
   adversaries.
   -

   *Shift the threat perception.* This is the hardest part: until the fear
   of a shared catastrophic outcome is as vivid as the fear of the rival, the
   race will continue. Perhaps that requires vivid scenario-building, or
   honest acknowledgment that a misaligned AGI is a common enemy that will
   treat all humans, regardless of nationality, as obstacles.

You’re absolutely right that normal human fear resists reasoned discourse.
And so the question becomes: *can we find a way to speak to fear in its own
language, and redirect it toward the outcome we all actually need to avoid?*

If not, then the article’s conclusion — that we must simply figure out
coexistence mid-flight, without a pause — may be the only path left, but
for reasons far darker than Morris acknowledges. Not because we chose it
wisely, but because our ancient fears gave us no choice at all.

--------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260610/08ad673a/attachment-0001.htm>